COMMUNICATION FROM THE COMMISSION - EU RESEARCH AND TECHNOLOGIGAL DEVELOPMENT ACTIVITIES: COMMISSION'S RESPONSES TO THE RECOMMENDATIONS OF THE INDEPENDENT EXTERNAL ASSESSMENTS OF THE LAST FIVE YEARS OF ACTIVITIES IN THE DOMAINS COVERED BY THE SPECIFIC PROGRAMMES AND THE JRC INSTITUTES UNDER THE FOURTH FRAMEWORK PROGRAMME AND EURATOM FRAMEWORK PROGRAMME
Date: 1997-04-16
Institutions: European Commission
Subjects: EU programme, Joint Research Centre, applied research, technological change

COMMISSION  OF THE  EUROPEAN  COMMUNITIES 

Brussels,  16.04.1997 
COM(97)  149  final 

COMMUNICATION FROM THE COMMISSION 

EU RJESEARCH AND TECHNOLOGICAL 

DEVELOPMENT ACTIVITIES: 

COMMISSION'S RESPONSES TO THE RECOMMENDATIONS 

OF THE 

INDEPENDENT EXTERNAL ASSESSMENTS 

IN THE DOMAINS COVERED 

BY 

THE SPECIFIC PROGRAMMES AND THE JRC INSTITUTES 

UNDER THE FOURTH FRAMEWORK PROGRAMME 

AND 

EURATOM FRAMEWORK PROGRAMME 

COMMISSION OF THE EUROPEAN COMMUNITIES 

Brussels,  16.04.1997 
COM(97)  149  final 

COMMUNICATION FROM THE COMMISSION 

EU RESEARCH AND TECHNOLOGICAL 

DEVELOPMENT ACTIVITIES: 

COMMISSIONS RESPONSES TO THE RECOMMENDATIONS 

OF THE 

INDEPENDENT EXTERNAL ASSESSMENTS 

OF THE LAST FIVE YEARS OF ACTIVITIES 

IN THE DOMAINS COVERED 

BY 

THE SPECIFIC PROGRAMMES AND THE JRC INSTITUTES 

UNDER THE FOURTH FRAMEWORK PROGRAMME 

AND 

EURATOM FRAMEWORK PROGRAMME 

INTRODUCTION 

The  Commission  welcomes  the  reports  resulting  from  the 5-year  assessment  exercises  carried  out by 
high  level,  independent  expert  panels.  The  Commission  appreciates  the  panels'  efforts  to  reach 
balanced  conclusions  and valuable  recommendations. 

The  Council  Decision  for  each  Specific  Programme1  requires  "an  external  assessment  conducted  by 
independent  experts  of  the  activities  carried  out  within  the  domains  covered  by  the  programme  and 
their  management  during  the  5  years  preceding  this  assessment".  To  this  end,  a  new  rationalized 
evaluation  approach  has  been  applied  which  complies  with  legislative  requirements,  takes  into 
account  advice  provided  by  CREST3,  and  follows  the  recommendations  of the Commission  initiative 
(SEM2000)  for  concrete  steps towards  best evaluation practices  across  the Commission's  Services4. 

During  the  second  half  of  1996,  18 expert  panels  have  assessed  RTD  efforts  sponsored  by  the  EU 
over  the  previous  5  years  under  the  various  Specific  Programmes.  In  addition,  7  independent 
visiting  groups  were  established  to  assess  the  Joint  Research  Centre  (JRC)  activities  (i.e.  JRC 
participation  in  Specific  Programmes  and  its  own  research  programmes)  and  another  high  level 
expert  to  provide  an  overall  evaluation  and  synthesis  of  the  visiting  groups  reports.  One  further 
group  was  charged  with  providing  an  assessment  covering  all  Community  RTD  activities  over  the 
last  5  years  (EC  and  Euratom  Framework  Programmes).  All  in  all,  some  170  senior  independent 
experts  have  been  involved  in the exercise,  which  subsumes  the  final  evaluations  of  the  programmes 
under  the  Third  Framework  Programme,  producing  18  Specific  Programme  reports,  7  JRC 
Institutes  reports  and  one global JRC  report,  and  one overall  Framework  Programme  report. 

This  Communication  presents  the  Commission's  responses  to  the  major  recommendations  provided 
in the  Specific  Programme  reports.  These  recommendations  are  included  in the executive  summaries 
which  are  reproduced  in  this  document  for  completeness.  The  opinions  expressed  in  the  executive 
summaries  are  those  of  the  external  expert  panels  and  are  given  under  their  responsibility.  (Each 
Specific  Programme  is presented  in a separate  section  including  three  parts:  I.  Executive  Summary; 
II.  Commission's  Response;  and  III.  Evaluation  Panel.  The  section  on  JRC  is  composed  of  the 
overall  evaluation  and  synthesis  report  and  the  list  of  experts  having  conducted  the  assessments  of 
the seven  Institutes.) 

The  Commission  notes  in  particular  the  comments  made  by  the  expert  panels  concerning  the 
management  and  the  procedures  of  RTD  programmes  and  will,  where  appropriate,  propose 
measures. 

The  full  expert  reports  for  the  Specific  Programmes  will  be  published  separately.  The  results  of  the 
JRC  assessments  together  with  the  comments  of  the  Board  of  Governors  will  be  communicated 
separately. 

The  Framework  Programmes  overall  assessment  together  with the  Commission's  responses  will  also 
be communicated  separately. 

1  Council  Decisions:  94/801/EC  of 23 November  1994; 94/572/EC  of 27  July  1994; 94/802/EC  of  23 November  1994; 94/571/EC  of  27 
July  1994;  94/803/EC  of  23  November  1994;  94/911/EC  of  15  December  1994;  94/804/EC  of  23  November  1994;  94/912/EC  of  15 
December  1994; 94/913/EC  of  15 December  1994; 94/805/EC  of 23 November  1994; 94/806/EC  of 23 November  1994; 94/914/EC  of  15 
December  1994; 94/915/EC  of  15 December  1994; 94/807/EC  of 23 November  1994; 94/917/EC  of  15 December  1994; 94/916/EC  of  15 
December  1994; 94/920/Euratom  of  15 December  1994; 94/799/Euratom  of 8 December  1994; and Council  Decisions  implemented  by  the 
Joint  Research  Centre: 94/918/EC of  15 December  1994; 94/919/Euratom of  15 December  1994. 

2  COM(96)22() final,  22 May  1996 
3 

CREST  advice  to Council  and  the Commission  on the  monitoring  and  evaluation  procedures  for  Community  research  programmes. 

4 

Communication  from  Commissioners  Liikanen  and  Gradin  in agreement  with  the  President,  SEC(96)659/5,  29 April  1996. 

'  For the JRC. the relevant  articles of the Decisions do not require  a response  by the Commission  to the experts'  reports. 

TABLE OF CONTENTS 

1. Telematics application 

2. Advanced communication telechnologies and services 

3. Information technologies 

4. Industrial and materials technologies 

5. Measurements and testing 

6. Environment and climate 

7. Marine science and technologies 

8. Biotechnology 

9. Biomedicine and health 

10. Agriculture and fisheries,  including agro-industry 

11. Non-nuclear energy 

12. Transport 

13. Targeted socio-economic research 

14. Cooperation with third countries 

15. Dissemination and optimization of results (Innovation) 

16. Stimulation of training and mobility 

17. Nuclear fission safety 

18. Controlled thermonuclear fusion 

19. Joint Research Center (JRC) 

Annexes: Terms of reference of the five-year assessment 

•  Annex 1: Specific Programmes 

•  Annex 2: JRC Institutes 

page 

7 

19 

27 

43 

55 

61 

73 

81 

91 

103 

Ill 

119 

125 

135 

147 

155 

161 

169 

177 

191 

192 

193 

1996 FIVE-YEAR ASSESSMENT 

TELEMATICS APPLICATION 

I. 

EXECUTIVE  SUMMARY 

Key  Conclusions 

1.  The  ICT  Programme  Structure.  A  single  Information  and  Communication  Technologies 
Programme  is  recommended,  with  a  major  section  of  it  devoted  to  Applications  Systems.  This 
would  continue  on  from  the  Telematics  Applications  Programme,  retaining  the  successful  features  of 
that programme,  notably  strong  user  involvement  in real  systems development  and  validation. 

2.  Vision  and  Strategic  Thinking.  The  Programme  needs  to  develop  stronger  vision  and  strategic 
thinking  to  concentrate  the  limited  resources  where  development  work  will  be  most  effective. 
Links with the Policy  parts of the  Commission need to be  strengthened. 

3.  The  Applications  Systems  Section  Structure.  The  worlds  of  multi-media  telematics  and  networks 
have  developed  considerably  since  the  Telematics  Programme  was  authorised  in  June  1991,  and  are 
continuing  to  evolve  very  fast.  The  transition  from  the  4th  Framework  Programme  (FP)  to  the  5th  FP 
presents  the  opportunity  to  rethink  the  applications  section  of  the  programme  in  the  light  of  these 
technological,  market  and  socio-political  developments.  The  applications  projects  should  be 
concentrated  on  the  emerging  sectors  of  new  telematics  use  and  on  those  where  there  is  a 
particularly  strong  economic  or  social  case,  such  as  those  thrown  up  by  new  developments  and 
those  related to the information  providers  part of the  "Information  Society". 

4.  The  Application  Systems  Section  Content.  In  addition  to  the  calls  for  applications  projects 
organised  on a Sector  basis,  a separate  series of  calls should  aim to create  integrated  systems  with 
joint  usage  of  two  or  more  different  applications,  breaking  down  the  Sector  boundaries.  In  order 
to  magnify  the  impact  of  the  programme,  a  very  important  part  of  the  work  of  all  Sectors  of  the 
Applications  programme  should  be  to  build  up  the  physical  and  intellectual  infrastructure, 
including  standards  work,  and  to  create  platforms  that  will  help  to  replicate  the  individual 
applications  projects.  Such work should be given priority throughout  the  Programme. 

5.  Administration,  etc.  The  "bureaucracy"  surrounding  this,  and  other  Framework  Programmes  has 
steadily  increased  to  the  point  where  key  firms  and  public  user  bodies  are  refusing  to  participate,  and 
where  it  is  irresponsible  to  encourage  the  participation  of  the  high  growth  potential,  small  firms,  and 
the  new  applicants  we  would  most  wish  to  see  in  the  programme.  Action  must  be  taken  to  improve 
delegation  and  reduce  nugatory  paper-work.  Participants  in  the  Programme  find  their  project  work 
inhibited  by  the endless demands  on their  time.  For  example,  measures  and projects  at both  Programme 
and  Sector  level,  designed  to  support  the  Programme  and  projects  as  a whole,  seem  to  the  participants 
to  have  risen  considerably  in  the  4th  FP,  creating  an  unfortunate  impact  in  the  time  and  effort  they 
cause  to  the  project  teams,  which  is unlikely  to be  balanced  by  the  benefits.  These  overhead  projects 
should  be  pruned  and  normally  established  only  at the  request  of  participants  in  the  applications 
projects. 

Introduction 

1.  Remit  of the Telematics  Applications  Programme  Five Year Assessment  Panel 

The  Telematics  Five-Year  Assessment  Panel  was  formed  early  in  1996 to cover  the  RTD  activities  carried 
out  in  the  field  of  "telematics"  (that  is  in  the  field  of  applications  of  computer  and/or  communication 
technologies)  during  the  period  from  1991  to  early  1996.  So  the  remit  of  the  Panel  subsumes  the  final 
review  of the Telematics  Programme  under  the 3rd  FP (Framework  Programme),  and the  mid-term  review 
under  the 4th  FP,  into this  5-year  assessment.  There  were  13 members  of the  Review  Panel  because  of  the 
complexity  and  wide  coverage  of  this  Telematics  Applications  Programme.  The  Panel  interviewed  the 
senior  staff  and  others  connected  with  the  running  of  the  programme,  and  various  individual  project 
participants.  Working  in  sub-groups,  Panel  members  examined  the  work  of  the  individual  sectors. 

Questionnaires  were  sent  to all the main participants  in the  3rd  FP projects  (1860) and  to the  co-ordinators 
of such of the 4th  FP projects  as had been authorised  at that time (370). 

1. 

2.  Telematics  Applications  Programme:  History 

The  Telematics  Applications  Programme  was  preceded  by  the  three  exploratory  actions,  established  in 
1989  as  part  of  the  2nd  FP.  The  Programme  was  established  as  part  of  the  3rd  FP,  covering  7  Sectors, 
with  a budget  of  430  Mecu.  Four  other  Sectors  were  added  in the  4th  FP,  with  a total  Programme  budget 
of  912  Mecu.  312 projects  were  funded  under  the  3rd  FP;  so far  422  projects  have  been  funded  under  the 
4th  FP. 

3.  Structure  of the Full  Report 

The  Full  Report  of the  Panel  consists  of the Key  Conclusions,  as given  above  in this  shortened  version;  an 
Introduction;  an  Executive  Summary  in  the  form  of  a  Discussion  of  the  Panels  Findings;  a  section 
presenting  and  explaining  the  Panels'  main  Conclusions  and  Recommendations;  and  a  set  of  the 
Conclusions  and  Recommendations  on  each  of  the  12  Sectors  of  the  Programme,  with  the  full  Sector 
reports  given  in an  Annex.  Other  Annexes  cover  the  analysis  of  the  questionnaires  and  finance  and  project 
statistics.  This  Shortened  Report  is based  on the  Discussion  section  of the  full  report;  the  reader  is  strongly 
encouraged  to  look  at  the  full  report  to  do  justice  to  the  conclusions,  and  to  the  recommendations 
stemming  from  those  conclusions  which  space prevents  setting  out  in this  Shortened  Report. 

Discussion  of  the  Key  Findings 

1.  The  Achievements 

lies  in  Applications; 

The  essence  of  the  Telematics  Applications  Programme 
the  applications  of 
information  and  communications  technology  to the problems  of  explicit  categories  of  users,  making  use  of 
systems  that  harness  emerging  technologies  of  networking  and  multimedia  to provide  solutions.  There  has 
been  a significant  shift  between  the  3rd  and  the 4th  FP  in the  extent  and  nature  of  the  user  involvement  in 
the  programme.  True  end  users  can  validate  the  system  approach  and  the  human  interfaces,  and 
intermediate  users  or  customer  bodies  can  control  or  at  least  influence  the  subsequent  deployment  of  the 
fruits  of  the  R&D  work.  The  Panel  found  that  the  majority  of  the  projects  in  the  Programme  are  firmly 
directed  to  the  solution  of  perceived  problems.  Especially  in  the  4th  FP,  users  are  strongly  involved  in 
drawing  up  the  workplans,  in  evaluation  of  the  applications,  and  in  the  conduct  of  the  approved  projects. 
This  is much  to  be  welcomed.  The  so-called  "Ten  Commandments"  of  the  Programme  have  proved  to be 
an  excellent  way  of  encouraging  highly  desirable  trends  throughout  the  Programme,  and  especially  in  the 
encouragement  of  user  involvement.  It  is too  early  to have  proof  that  it  increases  the  likelihood  of  project 
success.  However,  it  is  a  reasonable  expectation  that  true  involvement  of  users  in  the  programme  will 
improve  the deployment  of telematics  applications. 

It  has  been  the  great  achievement  of  earlier  generations  of  Commission  programmes,  like  ESPRIT  and 
RACE,  that  the  academic  and  industrial  research  workers  of  Europe  have  learnt  to  co-operate  together 
through  exposure  to  each  other  in  the  work  of  the  programmes.  Now  the  Telematics  Applications 
Programme  is pioneering  the  coming  together  of  the  users  of  Europe.  This  may  be  the  great  contribution 
of  the  Telematics  Applications  Programme.  Greater  coherence  in  the  users,  the  customers  of  ICT  in 
Europe,  will  make  for  faster  reacting  and better  informed  markets,  to the benefit  of all  concerned. 

2.  The Changed  Environment 

The  Telematics  Applications  Programme  has been  running  for  some  5 years.  During  the  last  few  years  the 
environment  for  telematics  applications  has  changed  dramatically  with  the  use  of  the  Internet  growing 
exponentially,  and  awareness  of  the  potential  benefits  of  telematics  becoming  widespread.  The  use  of 
multimedia  has become  common-place.  Prices  of  computer  hardware  and  the  cost  of  bandwidth  are  falling 
dramatically,  and  are  predicted  to  continue  to  do  so.  Of  course  the  penetration  of  multi-media  telematics 

10 

has  still  got  a  long  way  to  go.  The  challenge  now  is  both  to  accelerate  the  efficient  use  of  such  systems, 
especially  where  little  penetration  has  yet  been  achieved,  and  to  encourage  the  emergence  of  integrated 
applications. 

1. 

3.  Time  for  a  Rethink 

In  the  light  of  the  dramatic  changes  in  the  market  place,  the  Telematics  Applications  Programme 
sometimes  displays  a certain  tiredness.  While not throwing  away  the distinctive  and beneficial  features  of a 
programme  that  concentrates  on  real  applications  with  strong  user  involvement,  the  limited  resources 
must  be  used  in  a  way  that  maximises  positive  impact,  and  concentrates  on  the  areas  of  most  need 
and  where  the  time-scale  of  penetration  can be best  accelerated.  In what  is now  a global  market  place, 
only  quality  will  lead  to  success  so  projects  must  involve  the  European  bodies  most  capable  of 
producing  quality  work.  Some  restructuring  and  re-engineering  of  the  Programme  will  be  required, 
with the addition  to the current  main thrust,  using applications  projects,  of new  approaches  that  aim 
to accelerate  the take-up and integration  of telematics,  and its migration towards  multimedia. 

3.1  The  Opportunity  for  Europe.  The  emerging  multimedia  industries  will  include  some  of  the  largest 
business  opportunities  of the  next  two decades.  Yet,  there  is no clear picture  of how  this  futuristic  industry 
will  evolve.  In  the  presence  of  such  uncertainty,  players  who  articulate  a consistent  strategy  to  exploit  the 
opportunities  and  address  the  risks  will  shape  the  evolution  of  the  industry  structure.  Since  Europe 
currently  lags  well  behind  the  market  penetration  figures  for  the  United  States  of  America,  we  have  an 
unique  opportunity  to  develop  here  a  strong,  early  market  for  these  new  applications  -  upon  which  world 
wide  market  success  can be built.  The Applications  Programme  must be rethought  to take  into  account 
this  opportunity. 

3.2  Infrastructure,  Standards  and  Platforms.  It  is  important  to  ensure  that  the  necessary  infrastructure 
is  there  to  enable  the  current  market  expansion  to  continue  in  an  efficient  and  unfettered  manner. 
Infrastructure  is characterised  by  the  need  for  standards  in the broadest  sense;  standards  of performance  or 
quality  as  well  as  of  specification.  The  development  of  open standards  must  be  encouraged  to  prevent 
the  emergence  of  de  facto  standards  in  the  hands  of  the  few.  Steps  must  be  taken  throughout  the 
programme  to  emphasise  the  development  of  the  necessary  physical  and  intellectual  infrastructure. 
The  presence  of  many  users  in  the  programme  creates  a  unique  opportunity  to  do  this  effectively  if  the 
projects  achieve  the  right  balance  between  standards,  infrastructure,  and  applications  pilots.  There  is  a 
marked  trend  away  from  bespoke  systems  to  the  use  of  common  platform  systems  that  serve  a  class  of 
applications.  Ways  must  be  found  in  the  Programme  to  encourage  the  development  and  use  of  such 
platforms.  The  need  to  encourage  work  on  standards,  on infrastructure,  and  on  platforms  that  help 
to  multiply  the  impact  of  the  individual  projects  is  so  strong  that  consideration  should  be  given  to 
making  it a necessary  condition  of  every  project  application  that it has  at least  some  element  of  these 
properties  in it. 

3.3  New  Systems  for  the End-user.  Companies  round  the  world  are  working  on  new,  simple  devices  that 
are  aimed  at  making  it  easy  to  access  multimedia  applications.  With  such  an  "information  appliance" 
system  it  will  become  possible  to  bring  the  "unwired  masses"  into  the  Information  age.  New  information 
appliances  will  open  the  way  for  new  services  and  leisure  activities.  The  successful  information  appliance 
system  will  come  in many  flavours,  but whatever  form  it takes,  software  will be the  key  component.  Ways 
must  be found  in the new  Programme  to encourage the development  and usage of such  appliances. 

3.4  Focusing  and  Integration  of  Applications.  The  Programme  has  shown  a  welcome  flexibility  to  test 
and  then  implement  new  Sectors  for  new  constituencies  of users.  This  has  led  to  a growth  of  the  coverage 
of  the  Programme,  now  with  12  Sectors.  The  increase  in  the  number  of  Sectors  has  brought  about  a 
proliferation  of  Sector  advisory  committees,  workplan  working  parties  and  calls.  So,  though  it  is 
appropriate  to  continue  to  organise  much  of  the  applications  part  of  the  Programme  in  Sectors  with 
their  own constituencies,  the time has come to concentrate  the limited resources  on:  1) applications  in 
those  Sectors  where the  relative  immaturity  of  current  use  requires  attention;  and in those  few  other 
Sectors  where  there  is  particularly  strong  economic  or  social  benefit.  2)  the  integration  of 

11 

applications  arising  from  many  constituencies  into  common  systems  with  many  uses  and  diverse 
users.  This  is likely  to  require  special  calls,  like  the  one planned  for  "Integrated  Applications"  announced 
for  December  1996. And above  all: 3) the  development  of standards,  infrastructure,  and  platforms  to 
encourage  accelerated  take-up by the emergence  of common  approaches. 

3.5  Vision  and  Strategic  Thinking.  At  all  levels  of  the  Programme  there  is  a  need  for  constantly 
adapting  vision  and  strategic  thinking  to  ensure  that  the  fast  changing  external  scene  is  properly 
considered  to  direct  the  limited  resources  to  areas  and  in  ways  where  they  can  be  of  most  benefit. 
This  will  require  close  collaboration  with the policy  parts of the  Commission. 

1. 

4.  The ICT Programme  as a Whole 

4.1  A  Single  ICT  Programme.  For  years  there  has  been  a  degree  of  friction  between  the  three  ICT 
programmes.  This  has  not  proved  to  be  a  major  problem,  being  handled  by  good  co-operation.  In 
particular  the  co-operation  between  Telematics  and  ESPRIT  seems  to  have  improved  in  recent  years,  but 
the  arguments  for  creating  a  single  ICT  programme  remain:  1)  It  is  inevitable  that  there  has  to  be 
considerable  interaction  between  the  enabling  technology  and  the  systems  or  application  work,  and  gaps 
must  be  avoided..  Some  parts  of  the  subject  still  have  much  fundamental  work  to be  done  on  them.  There 
has to be strong  interaction  and  the boundary  will  shift  all the time.  2)  It  is undesirable  that  applicants,  and 
especially  new  participants,  should  be  confused  about  which  programme  they  should  be  participating  in, 
and  even  less desirable  that  they  should  play  off  one part  of the  staff  against  others.  To provide  coherence, 
clarity  of  vision,  and  drive,  a  single  programme  will  require  a  single  dynamic,  charismatic,  and  well 
informed  leader. 

4.2  A  Single  Management  Structure.  A  single  management  structure  should  be  created  for  the  5th 
FP.  The  multiple  committees  of  government  officials  or  nominees  do  serve  a valuable  role  in  keeping  the 
relevant  parts  of  the  national  governments  well  informed.  This  is  particularly  important  in  the  area  of 
applications.  By  increasing  the  coherence  between  the  sub-sections,  administrative  procedures  can  be 
improved.  The  temptation  to  impose  unnecessarily  uniform  procedures  should  be  resisted.  In  determining 
the  details  of  the  structure  one  good  guide-line  is to  consider  what  is the  primary  constituency  that  will  be 
making  applications  to  work  on  that  part  of  the  programme.  So  the  sub-sections  of  the  Programme 
might be: 

1) Long  Range  Research.  Essentially  the  old  "Basic  Research"  category  but  with  certain  additional  fields 
stemming  from  the  needs  of  applications.  This  would  be  the  constituency  of  the  advanced,  basic  ICT 
research  workers  in  the  academic  and  industrial  world.  The  need  for  long  range,  research  is  as  strong  as 
ever,  because  of the decline  of  support  in parts  of both the public  and private  sectors  in  Europe. 

2)  Enabling  Technology  (or  Applicable  Technology)  work.  Essentially  the  ESPRIT  and  ACTS 
programmes  where  the  constituency  is  primarily  the  ICT  industries,  but  without  their  more  user  oriented 
work  where  the users  come  primarily  from  constituencies  other  than  the  ICT  industry.  This  section  can  be 
seen as component  or sub-systems  work. 

3)  Applications  Systems  Development.  This  covers  user  oriented  work  for  clear  constituencies,  other 
than  from 
the  other 
the  ICT  industries;  the  Telematics  stream  with  user  oriented  streams  from 
programmes.  The work  of this sub-section  can be  seen as system  integration  and  proving  work. 

5.  Structure  of the Applications  Systems  Section  of the  Programme 

5.1  The  Need  for  Sub-Structure.  The  sub-structure  for  the  Applications  Systems  section  should  be 
based  on  natural  constituencies  of  users.  Cross  sector  and  integrated  applications  will  need  to  be 
encouraged.  It  is  not  desirable  to  divide  public  from  private  applications  in  conditions,  where  public 
bodies  can  become  private  bodies;  the  difference  is  not  significantly  larger  than  that  between  certain 
constituencies  of private  sector  users. 

12 

1. 

5.2  Reducing  the  Number  of  Sectors.  Too  many  sectors  creates  confusion  for  the  inexperienced 
applicant.  The  project  that  spans  the  boundaries  of  Sectors  is  less  likely  to  find  favour  with  the  selection 
panels  than  one  that  falls  squarely  in the  centre  of  one  Sector's  workplan.  Yet  there  are  reasons  to  expect 
that  a project  spanning  more  than  one  application  field  may  be  particularly  deserving  of  support.  So  there 
is every  incentive  to  reduce  the  number  of  Sectors.  But,  on balance,  it  is better  to  retain  sectors  for  users 
with  a  major  common  interest,  than  to  break  down  the  boundaries  and  create  a  continuum  with  no 
distinguishing  features.  However  there  should  be  calls  designed  to  transcend  the  Sector  boundaries.  The 
Panel  recommends  that  existing  Sectors  should  be merged  wherever  this makes  sense  in terms  of  the 
convergence  of  the  constituencies,  but  that  this  process  should  not  be  pushed  too  far.  The  primary 
consideration  must  lie  in  the  way  the  constituencies  see  themselves.  A  tightly  drawn  workplan  is  the 
best  way  to  reduce  the  over-subscription  problem,  provided  the  body  that  draws  up  a  workplan  is  well 
informed  about  the priorities  for  that constituency.  This mechanism  should  also handle  the  strategic  market 
survey  to detect  constituencies  that deserve  a new part  in the programme. 

5.3  Cross  Sector  Co-operation.  Steps,  such  as  special  calls,  have  to  be  addressed  to  encourage  cross-
sector  applications  and  merged  or  integrated  applications.  So the  Applications  sub-programme  can  be  seen 
as consisting  of  a number  of  Sectors  at  one  time  or  call,  but  as  a continuum  without  boundaries  at  another 
time  or  call.  The  majority  of  the  available  funds  must  still  be  devoted  to  the  normal  Sector 
applications systems projects,  but say  10 - 20%, should be reserved  for  cross  Sector calls. 

6.  The  Challenges 

The  challenges  for  the  5th  FP  Applications  Systems  Section  of  the  ICT  Programme  are  a mixture  of  those 
that  have  always  challenged  the  programme  managers  (such  as priority  and  criteria  setting,  the  evaluation 
process  in  a  multi-national  environment,  project  monitoring  in  a  constructive  and  creative  way,  and 
fostering  exploitation  while  maximising  actions  like  standardisation  work);  and  those  new  challenges  that 
now  face  the  Programme  manager.  Amongst  these are: 

6.1  User  Involvement.  The  involvement  of  users  is  a distinctive  and  successful  feature  of  the  Telematics 
Applications  Programme.  However,  the  involvement  of users  in projects  is not without  its problems: 

1) Users  are  frequently  inexperienced  in participation  in R&D  projects,  unaware  of  the  conventions  of  the 
"Peer  Review"  process.  Some  are  more  used  to political  methods  rather  than  the  scientific  approach.  This 
can  lead  to problems  when political  pressure  is applied  to influence  application  selection. 

2)  There  is  also  the  danger  that  the  users  will  be  encouraged  to  become  research  workers  themselves, 
which  can  lead  to poor  project  performance. 

3) If the users  get  too much  influence  over  the workplan  and project  selection  process  they  can be  resistant 
to the desirable  adoption of  innovative  new technologies  and forward-looking  approaches. 

4)  Most  pilot  users  are  large  organisations  who  may  wish  to  tailor  the  application  too  tightly  to  their 
specific  needs,  leading  to strong  risks of over-specification  with resulting  limited  transferability. 

Some ways of  minimising  these problems  are  suggested  in the main  report. 

6.2  Administrative  Burden 

1)  In  the  Process.  It  is  inevitable  that  programmes  deploying  public  funds  will  be  hedged  about  with 
regulations  designed  to  safeguard  the  public  purse.  But  these  have  steadily  grown  over  recent  years  to  the 
point  where  the  programme  is  being  strangled.  Many  private  sector  companies  are  no  longer  prepared  to 
participate.  The  costs  of  preparing  proposals,  the  delays  inherent  in  project  approval  and  contract 
finalisation,  combined  with  the  relatively  small  proportion  of  applications  that  can  be  funded,  make  it 
preferable  to forego  the  advantages  in participation.  Delegation must be improved. 

13 

1. 

2)  In  the  Project.  Once  a  proposal  has  been  approved  and  a  contract  signed,  enabling  the  work  to 
proceed,  the  research  worker  finds  an  ever  increasing  burden  associated  with  the  project  reporting  and 
concertation  process,  etc.  The  Commission  senior  staff  claim  that  some  aspects  of  the  burden  has  been 
reduced  in the  4th  FP,  but  the  workers  seem  to perceive  an  overall  increase,  perhaps  due  to  the  spread  of 
horizontal  projects.  These  distractions  represent  a  severe  overhead,  involving,  as  they  inevitably  do,  the 
time of senior  project  staff.  Measures must be taken to reduce such an overhead. 

The small  high technology  firm  or the applicant  new to the programme  are  likely  to find  the overheads  and 
delays  are  unacceptable.  Small  firms  cannot  afford  to  tie  up  their  scarce  resources  of  high  quality 
manpower  in  the  overhead  activities  required  of  a  participant  in  the  Framework  Programme  projects  as 
they  are  currently  organised.  It  is  irresponsible  to  encourage  such  a  firm  to participate.  Some  suggestions 
for  tackling this major challenge of reducing bureaucracy in the Programme are set out in the main  report. 
It is not,  of course,  a problem  restricted  to the Telematics  Applications  Programme. 

7.  Relations  with the Policy  DGs 

It  is highly  desirable  that the  relevant policy  DGs should take an active  interest  in and be closely  associated 
with  the  relevant  work  in an  Applications  Programme.  Special  mechanisms  been  established  for  certain  of 
the  Sectors.  These  measures  are  much  to be  welcomed,  and  such mechanisms  should  be  provided  to  all 
Sectors where  policy  DGs with an interest in the work are to be  found. 

14 

I I. 

COMMISSION'S  RESPONSE 

The  Commission  welcomes  this  independent  evaluation  of  RTD  activities  on  applications  of  information 
and  communication  technologies  in  areas  of  general  interest.  The  recommendations  will  be  taken  into 
account  in  orienting  the  further  development  of  technologies  in  the  Fourth  and  Fifth  Framework 
Programmes  and  in related  standardisation  and deployment  initiatives. 

information  and  communication 

Since  telematics  RTD  activities  were  started  at  the  European  Union  level  in  the  Third  Framework 
Programme, 
technologies  have  developed  considerably  beyond 
recognition  (e.g.  Internet,  CD-ROMs)  and  their  growing  convergence  implies  that  developments  will 
steadily  continue.  Hence,  the  transition  from  the  fourth  to  the  Fifth  Framework  Programme  presents  the 
opportunity  to  reconsider  the  Applications  component  of  any  future  programme  concerning  information 
society  related  research,  in  the  light  of  these  technological,  socio-economic  and  regulatory  developments. 
The  report  of the  five-year  assessment  panel  offers  very  useful  paths  to explore  in order  to focus  the  future 
activities  on  those  areas  which  have  the  strongest  European  added  value  and  where  there  is  a  genuine 
economic  and  social  case. 

As  the  report  has  clearly  noted,  during  the  3rd  Framework  Programme  the  RTD  work  was  devoted 
essentially  to  the  development  of  solutions  for  the  modernisation  of  services  of  general  interest  in  Europe. 
The  4th  Framework  Programme  has  brought  further  major  objectives,  notably  to  prepare  the  ground  for 
deployment  through  public-private  and public-public  partnerships,  and  to encourage  the  opening  up of  new 
markets  and  the  emergence  of new  companies,  in particular  innovative  SMEs. 

to  multi-media 

telematics:  from 

The  report  correctly  notes that the transition  from  the 3rd  to the 4th  Framework  Programme  included  three 
major  shifts:  from  the  linear  model  to  the  feedback  model  for  research,  technological  development  and 
demonstration;  from  data  telematics 
to  user-led 
orientation.  On  the  latter  point,  the  report  stresses  the  essential  contribution  made  by  telematics  RTD 
activities  in  developing  discreet,  topic  specific,  User  Constituencies.  Such  shifts  might  have  deserved  a 
stronger  emphasis  in the  report  as they  resulted  in dramatic  changes  in the work  content  and  in  programme 
and  project  management.  They  should  be  re-inforced  in  any  subsequent  work  in  the  Fifth  Framework 
Programme,  along  with  other  shifts,  e.g.  increased  concentration  of  RTD  effort,  stronger  contribution  to 
major  EU  policy  objectives,  full  implementation  of  the  "contextual  research"  model  (interactive  vs. 
linear),  systematic  involvement  of  users  in  projects  for  continuous  socio-economic  feedback,  more 
functional 
comprehensive  coverage  of  all  major 
specifications,  demonstrator  building, 
trials, 
through 
exploitation  plan containing  quantifiable  economic  criteria  for  future  assessment  of success). 

techno-economic  stages  (analysis  of  user  needs, 
laboratory  verification,  demonstration 

technology-push 

field 

The  Commission  welcomes  the  Panel's  view  that  separate  series  of  calls  for  proposals  should  aim  at 
creating  integrated  systems  with joint  usage  of  two or  more  different  applications,  with  a view  to  breaking 
down  the  sector  boundaries.  Already  under  the  Fourth  Framework  Programme,  the  call  to be  launched  in 
December  1996  on  integrated  applications  should  allow  the  Programme  to  draw  useful  lessons  on  the 
interest,  relevance  and  feasibility  of  such  an  approach  prior  to  implementing  the  next  Framework 
Programme.  These  integrated  applications  are  expected  to  provide  user  friendly  access  to  various 
applications  from  existing  platforms  (PC,  network  computer,  Kiosks,  personal  digital  assistant,  etc.),  to 
have  an  appropriate  scale  demonstration  phase  on  "digital  sites"  to  allow  significant  conclusions  to  be' 
drawn  in socio-economic  terms,  and  to enable  faster  move  towards  the  Information  Society.  The  results of 
this  call  and  the  continuous  assessment  of  project  implementation  will  provide  essential  information  in 
support  of  the  preparation  of  a  larger  scale  operation  for  integrated  applications  in  the  Fifth  Framework 
Programme. 

The  Commission  takes  note  of  the  Panel's  recommendation  to  focus  the  work  throughout  the  Programme 
on  building  up  the  physical  and  intellectual  infrastructure, 
including  standardisation  work  involving 
industry,  and  to  create  "platform  systems"  that  can  be  tailored  to  the  individual  user  requirement  whilst 
allowing  the  replication  of  applications  across different  sectors.  The use  of  such  specific  industry  platforms 
will  be  encouraged  in  further  RTD  activities,  drawing  usefully  on  the  lessons  learned  from  both  the  on-

15 

1. 

going  sectorial  projects  and  the  future  integrated  applications  projects.  This  approach  should  certainly 
contribute  to  encouraging  cross  sector  cooperation,  making  any  further  programme  function  as  a 
continuum  of activities  without  boundaries  instead  of as a collection  of distinct  sectors. 

The  Panel  recommendation  that  preparatory  accompanying  and  support  measures  as  well  as  "horizontal" 
RTD  activities  within projects  like those  of the  Telematics  Engineering  sector,  deserve  a careful  study  will 
be  taken  up.  In the  meantime,  the  Commission  does  consider  these  as  important  actions  for  supporting  the 
RTD  projects  and  for  fostering  the  standardisation  process  and  the  dissemination  and  innovation  process. 
In order  to avoid  their  becoming  what  the report  calls  "overheads  projects",  their  optimal  weight  should of 
course  be  assessed,  including  through  consultation  of  consortia  involved  in  RTD  projects,  and  their 
selection  be decided  against  a clearer  strategy  and transparent  indication  of the expected  benefits. 

The  Commission  shares  the  conviction  that  greater  coordination  between  RTD  programmes,  in  particular 
through  the  concept  of  research/industry  Task  Forces,  and  greater  flexibility  in  programme  management 
will  be  beneficial.  The  "bureaucracy"  as  it  is  identified  both  in  the  programme  management  process 
(delays  inherent  in project  approval  and  contract  finalisation,  large  oversubscription,  model  contract  based 
on  cost  statements,  etc.)  and  in  the  management  of  projects  (reporting,  "concertation",  etc.),  has  indeed 
been  increasingly  perceived  by  participants  across  the  specific  programmes  as  a  source  of  concern  and 
even  sometimes  a  source  of  insurmountable  difficulties  for  the  management  of  Consortia.  The  first 
orientations  for  the  Fifth  Framework  Programme,  proposed  by  the  Commission  under  the  title  "Inventing 
Tomorrow",  introduces  a  simpler  structure  and  more  flexible  management  ideas  consistent  with  the 
recommendations  of the  5-year  assessment. 

16 

I I I. 

EVALUATION  PANEL 

Mr.  Knud  CHRISTENSEN 
Director,  R&D  and  Production,  Scandinavian  Mobility  International  A/S.  Vice-Chairman,  Centre  for 
R&D  within  Rehabilitation  (1988-1990).  Management  consultant  with  expertise  in  strategic  planning, 
productivity  and  logistic  structures  (1988-1990). 

Dr.  Max  HERRY 
Owner  and  Managing  Director  of Traffic  Planning  Office  Herry,  Vienna,  and  of INVOPLAN,  Munich.  20 
years  experience  of  national  and  international  transport  and  telematics  projects.  Evaluator  of  4th  FP 
Transport  programme  (DG  VII).  President  of  DANUBE  (Exchanges  between  universities  and  industry  in 
the EU).  Project  leader  or consortium  member  of 7 finished  EU projects,  4 current  EU  projects. 

Prof.  Hannu  JAAKKOLA 
Professor  for  information  technology  at  Tampere  University,  Finland.  Director  of  research  institute  for 
information  technology  (1986-1992).  Author  of  some  80  publications  on  information  technology  and 
technology  management. 

Prof.  Dr.  Gunter KROËS 
Professor  of  national  economics,  Faculty  of  Urban  &  Regional  Planning,  University  of  Dortmund.  Head 
of  Department  for  National  Economies.  Expert  on  regional  development  in  rural  areas.  Chairman  of  the 
Telematics  Applications  programme's  external  monitoring  panel  in  1995. 

Dr.  Philippe  LAREDO 
Associate  Research  Director,  Centre  de  Sociologie  de  l'Innovation  (CSI)  at  the  Ecole  des  Mines  de  Paris 
(since  1987).  Responsible  for  research  in  programmes  in  transport  economics,  Ministry  of  Transport, 
France  (1979-1982).  Responsible  for  national  programme  on  marine  technologies  and  then  head  of 
evaluation  unit,  Ministry  of  Research  and  Technologies  (1983-1989).  Evaluation  of  many  EU  and  Eureka 
programmes.  Expert  with  numerous  papers  on R&D programmes  evaluation. 

Dr.  Louis  LEE 
Director  of  information  resource  services,  University  of  Westminster  (since  1989),  with  responsibility  for 
strategic  planning,  technology  and  development  of  academic  computing,  audio  visual  resources,  libraries 
and  management  information  services  to  the  University.  Non-executive  Director  of  an  SME  that  supplies 
library  systems  and  services.  Expert  on the use of telematics  in the university  management  environment. 

Dr. Jim  MOUNTJOY 
Managing  Director,  Euristix  Ltd.  Member  of  the  Irish  National  Information  Society  Steering  Committee. 
Former  member  ESPRIT  Management  Committee.  Member,  Telecommunications  Senior  Officials 
Committe  and  of  European  Space  Agency. 

Dr.  Brian OAKLEY  (Co-chairman  and secretary  of the panel) 
Retired.  Chairman  of  Logica,  Cambridge,  (1987-1993).  Director  of  the  Alvey  programme  (1983-1987). 
Secretary  of  Science  &  Engineering  Research  Council  (1979-1983).  Member  of  ESPRIT  Management 
Committee  (1983-1987).  Member  of  ESPRIT  Advisory  Committee  (1987-1992).  Chairman  of  Telematics 
programme  mid-term  review  (1993). 

Prof.  Torleif  OLHEDE 
Director  of  the  Department  of  Health  Informatics  at  Spri,  the  Swedish  Institute  for  Health  Services 
Development.  Research  Associate  at  the  Department  of  Computer  and  System  Sciences  at  Stockholm 
University  and  the  Royal  Institute  of  Technology.  Expert  in  security  informatics.  Swedish  representative 
on  Health  Care  Sector  Working  Party.  Head  of  group  responsible  for  armed  forces  strategy  for  IT, 
communications  and  computer  security  (1984-1989). 

17 

1. 

Dr. Hans-Detlef SCHULZ 
Managing  Director,  Media  Transfer  GmbH,  Darmstadt,  Germany.  Member  of  Telematics  programme 
mid-term  review  panel  (1993).  Panel  rapporteur,  4th  FP  call  for  proposals.  Expertise  in  the  field  of 
telecommunications  and  telecommunications  standards.  Rapporteur  for  multi-media  standards  in  ITU-T, 
member of ETSI, ISO, and DA VIC. 

Prof. Vincenzo TAGLIASCO 
Full  Professor  in  medical  informatics  at  the  University  of  Genova,  Italy.  Served  as  expert  for  the 
Commission  on studies  of  impact  of  EU  R&D programmes,  mid-term  evaluation  of  Human  Capital  and 
Mobility programme.  Expert and writer on the future  socio-economic  impact of demographic  change and 
technology development. 

Dr. Georges WANET 
Member  of  the  Executive  Committe  of  Belgacom  since  1992.  Head  of  Corporate  Strategy  and 
Development.  Was responsible  for  strategic planning,  data processing,  and the  FAST project  to  improve 
efficiency  of basic telephony  services. Also Director of F.N.A., grouping  12 telecommunication  operators 
to  provide  managed  bandwith  services  to  the  Financial  Community.  Member  of  Telematics  programme 
mid-term review panel (1993). 

Mr. Nils WILHJELM (Co-chairman of the Panel) 
Chief  Executive  of  Industriens  Realkreditfond  (since  1990).  Minister  for  Industry  and  Trade,  Danish 
Government  (1986-1989).  Chairman  of  Federation  of  Danish  Industries  (1983-1986).  Deputy  Chairman, 
Board of Directors of Tele Danmark A/S. 

1996 FIVE-YEAR ASSESSMENT 

ADVANCED COMMUNICATION TECHNOLOGIES 

AND SERVICES 

19 

T.  EXECUTIVE  SUMMARY 

Europe  has  excellent 
first-generation 
digital 
communications, 
and  a  European 
information 
infrastructure  is now 
urgently  needed. 

With  the  Euro-ISDN  and  GSM  standards,  Europe  has  an  excellent  set 
of  first-generation  digital  communications  infrastructures. 
In  terms  of 
the technologies  used,  they  are  as good  as any  in the  world.  However, 
it 
to  develop  communication 
technologies,  standardise  for  interoperation  and  introduce  a  European 
Information 
telecommunications,  data 
networking  and broadcasting  capabilities  as quickly  as possible. 

Infrastructure  combining 

to  continue 

for  Europe 

is  vital 

Digital  video  services 
should  be  introduced 
as quickly  as 
possible.. 

In  particular,  digital  video  services  should  be  introduced  as  quickly  as 
possible.  The  technology  for  digital  video  and  television  now  exists, 
and  a  programme  to  accelerate  the  penetration  of  digital  services 
would liberate valuable spectrum for  other services. 

Specific  features  of  a  European  Information  Infrastructure  need  also  to 
be  addressed  at  European  level,  taking  account  of  the  increasing 
importance  of  the  developing  Eastern  European  markets  arid  the 
Mediterranean  Countries. 

with a coherent  plan 
and  timetable for  the 
introduction  of  a 
European 
Information 
Infrastructure. 

The  cost  then  of  providing  broadband  services  to  every  household  in 
the  EU  alone  will  be  enormous.  Some  progress  has  been  made  in  the 
installation  of  the  core  network,  but  a  lot  needs  still  to  be  done  in 
extending  this  to  all  citizens  and  small  businesses.  Europe  badly 
needs  a  coherent  plan  and  timetable  for  the  introduction  of  a 
European Information  Infrastructure. 

The key  objectives  of 
the European  R&D 
have  been  met,  but 
coherent  deployment 
of  multimedia 
services is  slow. 

the  European  R&D 

The  key  objectives  of 
telecommunications 
programmes  (RACE  I /  II)  have  been  met.  The  programme  has  met  its 
technological  objectives,  but  the  task  of  technology  development  is  far 
from  being  completed. 
The  ACTS  Programme  has  created  an 
awareness  of  the  vast  potential  of  digital  technology.  However,  the 
deployment  of  advanced  multimedia  telecommunication  services  on  the 
European  market  has  not  reached  the  level  which  could  have  been 
expected. 
and 
interoperability,  but  legislation  may  be  necessary  in  some  cases  to 
ensure  coherent  deployment.  It  is  also  necessary  to  improve  access  to 
venture-capital  support  for  innovative  high-tech  enterprises  and  start 
ups. 

The  EU  RTD  has  helped 

interconnectivity 

Further  research 
and  technology 
development  is 
necessary 

Further  research  and  technology  development  is  necessary:  It  will  not 
be  possible  to  realise  an  Information  Society  in  Europe  only  with  the 
technologies  of  today,  and  technology  development  must  occupy  a  key 
position  in future  EU RTD.  Preference  should  be given  to projects  that 
lead  to  improvement  of  an  EII  and/or  to  future  concrete  commercial 
applications,  and  should  mainly  include  projects  that  could  have  major 
impact  on the Union  and  its population. 

21 

but  the focus  must 
now  shift  from 
technical  standards 
alone,  to trials and 
volume  deployment 

The  original  rationale  for  European  support  to  RTD  in  this  area  still 
applies,  but  the  focus  must  now  shift  from  defining  technical  standards 
alone,  to  enabling  applications  and  volume  deployment  on  the  basis  of 
unique  and  widely  accepted  standards.  European  standards  are  a 
strategic  means  to maintain  the technological  initiative  in Europe  and  to 
It  is  also 
get  a  better  industrial  position  in  the  global  marketplace. 
essential 
the 
realisation  of  new  developments  be  shortened.  Mechanisms  must  be 
found  to achieve  real market  impact. 

time  span  between  conceptualisation  and 

that 

the 

EU  RTD 
management  needs 
improved 
contracting.. 

In  terms  of  the  management  of  EU  RTD,  contracting  procedures  need 
to be  improved  and  the benefits  of using  a results-based  contract  should 
be  reviewed,  with  more  emphasis  on  technical  achievement  than  on 
accountancy.  Contracting  should  be faster  and more  transparent. 

good  programme 
integration... 

Independent  advice  on  programme  integration  and  management  should 
continue  to  be  sought,  and  more  use  should  be  made  of  video 
conferencing. 

across  programmes. 

Stronger 
links  should  be  established  with  the  EU  Training  and 
Mobility,  ESPRIT  (information  technology)  and  Telematics  Application 
programmes.  There  is  a  need  to  enhance  the  transparency  and  co 
operation, 
these 
programmes. 

including 

potential 

projects, 

between 

joint 

and  with  enhanced 
National  initiatives. 

And  the  concept  of  National  Hosts  needs  to be  re-evaluated  to  enhance 
their use. 

In  the  5th  F.  P., 
there  should  be a 
focus  on  the 
Communication  and 
Information  Society, 
with a  coherent 
strategy and  more 
flexibility. 

Work  in the  5th Framework  Programme  should  be  around  the theme  of 
RTD  for  the Communication  and  Information  Society.  There  must be a 
coherent  overview  of  the  whole  information  and  communications 
technology  development  area,  with  a  single  coherent  EU  strategy,  a 
single  strategic  advisory  group  and  flexibility  to  move  resources  within 
the  whole  area  as  circumstances  change.  One  possibility  might  be  a 
Core  Technology  Programme  and  Focused  Programmes  in  specific 
application  areas,  as described  in Section 6.3. 

The  main  objectives  should  be  to  overcome  present  technological  and 
infrastructural  constraints  to  the  explicit  and  hidden  needs  of  users  and 
to  stimulate  trials  to  demonstrate  "in-the-field"  the  effectiveness  for 
users  of  proposed  solutions. 
It  is  recommended  that  use  of  trial 
infrastructures  be enhanced  in each Member  State. 

Key requirements  in 
communications  will 
be seamless  access, 
ease of  access, 
quality  of  service, 
ubiquitous  access 
and  service 
customisation.. 

To  enable  market  development,  meeting  users  requirements,  the  key 
requirements  are  for  seamless  access;  ease  of  access;  quality  of 
services;  ubiquitous  access,  and  service  customisation. 
Particular 
efforts  must  be  made  to  improve  access  facilities,  improve  connection 
facilities,  and  optimise  the  interaction  between  the  "access  intelligence" 
and  the  "network  intelligence".  Standardisation  -  first  for  parts  and 
then  the  whole  -  should  remain  the  primary  task:  ATM  systems  must 
also  be  completely  and  finally  standardised,  and  multi-media  standards 
must be developed jointly  by various  standardisation  groups. 

22 

Enabling 
technologies and 
visionary concepts 
must both be 
covered. 

Other  key  areas  are 
in  optoelectronics/photonics,  microwave 
propagations,  software,  interactivity,  service  engineering,  intelligence 
in  networks, 
in 
information  security. 

intelligent  agents,  mobile  communications  and 

But  visionary  concepts  must  also  be  addressed,  some  examples  of 
which are given in Section 6.5. 

Recommendations 

1. 

Europe  needs  a  coherent  European  Information  Infrastructure  covering  all  new  communication 
requirements  emerging  from  telecommunications,  data  networking  and  broadcasting;  it  is  the 
responsibility of the Commission to take the necessary initiatives. 

2. 

To  reach  this goal,  it  is necessary  that  the process  of  liberalisation  in the  telecommunications  and 
broadcasting infrastructures  and services is speeded up. 

3.  As  a  counterpart,  it  is  also  necessary  to  strengthen  and  accelerate  standardisation.  Only  unique 

European standards will protect the investments of industry and consumers. 

4.  Ways must be found  to accelerate the volume deployment of digital TV and  interactive  multi-media 
services.  In addition, opportunities must be sought for exploitation of global niche markets for high-
value business services. 

5.  Access  to  financing  for  innovation  must  be  improved.  The  Commission  should  take  initiatives  to 
create a better venture-capital  financing  environment;  some options are indicated  in Section  1  of the 
Report. 

6.  A flexible  and functionally  effective  approach to EU RTD is needed  from  all European  Institutions. 
The Commission  must ensure co-ordination,  co-operation  and transparency  in their  management of 
programmes.  In advanced  communications,  good programme  integration  should  be continued,  and 
with more co-operation with other programmes. 

7.  We must prepare  now for  the 5th Framework  Programme,  with appropriate  and flexible  provisions 
for  Information  and  Communications  Technology  RTD;  some  of  the  key  issues  are  highlighted  in 
this Report and exploratory work in these areas should already be started. 

8. 

It  is vital  to include visionary  concepts,  as a means of enhancing  value  chains,  and  as the  seeds of 
future  European competitiveness. 

23 

II,  COMMISSION'S RESPONSE 

The Commission  welcomes  this  independent  evaluation of EU  RTD  in the broader  perspective  of  advanced 
communications  developments  in Europe.  The  recommendations  will be taken  into account  in orienting  the 
further  development  of  technologies  in  the  4th  and  5th  Framework  Programmes  and  in  related 
standardisation  and  investment  stimulation  initiatives. 

The  Commission  accepts  its  responsibilities  for  catalysing  the  development  of  a  coherent  European 
Information  Infrastructure  covering  all  new  communications  requirements  and  addressing  the  specific 
features  of  European  regions,  Eastern  European  and  Mediterranean  countries  markets.  The  technology 
developments  in the  RACE  (1988-1994)  and  ACTS  (1994-1998)  programmes  are directed  to support  inter 
operability  in  the  convergence  of  telecommunications,  data  networking  and  broadcasting  areas.  In 
for 
addition,  coherence  and 
communications  in the Information  Society. 

in  the  new  regulatory  framework 

interoperability  are  key  features 

Liberalisation  and  competition  in  provision  of  infrastructure  and  services  has  been  a  key  objective  of  EU 
telecommunications  policy  in  recent  years.  The  Commission  will  seek  to  ensure  that  the  process  of 
liberalisation  in telecommunications  is followed  through by Member  States on the agreed  timetable. 

The  multiplicity  of  infrastructure  and  service  providers  in  a competitive  market,  and  the  new  diversity  of 
technologies  for  service  provision,  will  increase  the  need  for  effective  standardisation  and  consensus 
development.  The  EU  RTD  programmes,  both  RACE  and  ACTS,  have  made  major  contributions  to 
technical  standardisation  -  as  noted  in  the  5-year  assessment.  Further  support  will  be  provided  from  the 
remainder  of  the  ACTS  programme 
industry-led 
standardisation.  The  need  for  substantial  EU  support  to standardisation  will be  considered  in preparing  the 
Commission's  proposal  for  Information  Society  RTD  in the 5th Framework  Programme. 

in  the  form  of  guidelines  and  contributions  to 

The  volume  deployment  of  digital  TV  and  interactive  multi-media  services  is  primarily  a  responsibility  of 
the  private  sector.  Nevertheless,  the  new  framework  for  EU  investment  stimulation  in  Transeuropean 
Networks  will  be  used  by  the  Commission  to  catalyse  the  coherent  and  simultaneous  roll-out  of  new 
services  within  the  EU.  Exploitation  of  global  niche  markets  for  high-value  business  services  will  be 
facilitated  by  emerging  "electronic  commerce"  systems.  The  Commission  will  implement  a  new  Co 
ordinated  Electronic  Commerce  Initiative  in  1997,  building  on  the  RTD  in the  ACTS  programme  and  the 
Memorandum  of  Understanding  recently  signed by leading  industrial  interests  and  SME  associations. 

In  1996,  the  Commission  published  a Green  Paper  on  Innovation,  and  has  adopted  an  Action  Plan  on  the 
basis  of  the  reactions  from  Industry  and  Member  States.  The  Green  Paper  covered  the  improvement  of 
innovation  financing,  particularly  for  new  technology-based  firms.  The  options  in  the  Strategic  Audit 
Report  will be given careful  consideration. 

The  Commission  shares  the  conviction  that  greater  co-ordination  between  RTD  programmes,  and  greater 
flexibility  in  programme  management  will  be  beneficial.  The  first  orientations  for  the  5th  Framework 
Programme,  proposed  by  the  Commission  as  "Inventing  tomorrow"  ,  introduce  a  simpler  structure  and 
more  flexible  management  ideas  consistent  with  the  recommendations  in  this  "Strategic  Audit".  A  re 
inforced  provision  for  visionary  research  at European  level will  allow  many  of  the  new  ideas  in this  report 
to be taken  forward. 

COM(96)332 Final of  10.07.96. 

24 

Ill,  EVALUATION  PANEL 

Mr.  Stelios Argyros  MEP 
As  a  Member  of  the  European  Parliament,  Stelios  Argyros  sits  on  the  Committee  on  Research, 
Technological  Development  and  Energy  and  the  Committee  on  Economic  and  Monetary  Affairs  and 
Industrial  Policy.  He  has  served  on  the  Board  of  numerous  companies  in  Greece,  has  been  Chairman  of 
the Board  and  President  of the  Federation  of Greek  Industries  and  is currently  a Vice  President  of  UNICE. 

Mr.  Max  Artigalas 
General  Manager  Recording  Programme  Administration  of THOMSON  multimedia.  After  being  technical 
manager  of  Thomson  Broadcast,  he  joined  THOMSON  multimedia  where  he  is  also  recording  domain 
manager  for  Corporate  Research  and  Advanced  Development. 

Dr.  Hans  Baur 
Former  Executive  Vice  President  of  Siemens  AG,  and  now  acting  as  an  advisor  to  several  national  and 
international  bodies  and  to  Mr.  Bangemann  on  the  Information  Society.  During  his  career  with  Siemens 
AG,  Dr.  Baur  was  a  member  of  the  Corporate  Executive  Office  and  was  responsible  for  the  business 
groups  dealing  with  telecommunications  and  electronic  components.  He  also  served  as  President  of  the 
Supervisory  Board  of  Siemens  Nixdorf  Informationssystems  AG. 

Mr.  Umberto  de Julio 
Co-General  Manager  of  STET,  with  responsibility  for  strategic  planning,  technology  and  development  of 
new  service,  prior  to joining  STET  in  1995, Mr.  de  Julio  held  the  position  of  Director  of  the  Network  of 
Telecom  Italia  in  1994.  His  earlier  career  with  SIP  began  in  1972,  where,  in  1986,  he  was  appointed 
Director  of  the  Network.  He  is a Member  of the Board  of Telecom  Italia,  Telecom  Italia  Mobile,  CSELT, 
STREAM,  STET  International  and  is  also  President  of  the  General  Assembly  of  EURESCOM  and  Vice-
General  President  of  AEI.  Mr.  de  Julio  was  assisted  by  Mr.  Giovanni  Perucca,  Director  of  R&D  of 
STET. 

Mr.  Aimo  Eloholma 
Vice  President  of  Telecom  Finland  Ltd.,  Sales  and  Marketing,  since  1995,  Mr.  Eloholma  was  formerly 
Vice  President  of  Telecom  Finland,  Business  Services,  and  has  held  a variety  of  positions  within  the  PTT 
in Finland.  He  is Chairman  of the  Board  of Tele  Yritysviestintà  Oy  (provider  of  PBX's)  and  Yritysverkot 
Oy  (Corporate  Networks  Ltd.)  and  serves  on a number  of Committees  in Finland.  He  is also  a member  of 
Information  Society  Forum  (European  Commission).  Mr.  Eloholma  was  assisted  by  Mr.  Timo  Rajamaki, 
Director  of the  Research  Centre  of Telecom  Finland. 

Dr.  Francisco  Pinto Balsemâo 
Chairman  and  majority  share  holder  of  IMPRESA,  S.G.P.S.,  a  holding  company  with  major  positions  in 
newspapers,  printing,  advertising  and  a  Portuguese  private  television  channel.  He  is  Professor  of 
Communication  Science  at  the  New  University  of  Lisbon,  Chairman  of  the  European  Institute  for  the 
Media,  Vice-President  of  "Fondation  Journalistes  en  Europe"  and  Member  of  the  Board  of  the  Media 
Business  School.  He  was  one  of  the  founders  (1974)  of  the  Popular  Democratic  Party  (PPD),  now  the 
Social  Democratic  Party  (PSD)  and  from  1981  to  1983  he  was  Prime  Minister  of  Portugal.  Dr.  Pinto 
Balsemâo  was assisted  by Mr.  Antonio Trigo de Sousa,  Technical  Director  in SIC. 

Dr.  George  T.  Waters 
Director  of  the  Technical  Department  of  the  European  Broadcasting  Union  since  1986,  Dr.  Waters  has 
been  involved  in  broadcasting  for  more  than  35  years.  He  completed  a  seven-year  term  as  Director 
General  of  RTE,  Ireland's  national  broadcasting  organisation,  and  was  a  Director  of  the  International 
Council  of  the  Academy  of  Television  Arts  and  Sciences  (USA)  and  Vice  President  of  the  European 
Broadcasting  Union.  In  1993  he  was  appointed  President  of  the  International  Academy  of  Broadcasting 
(IAB). 

25 

1996 FIVE-YEAR ASSESSMENT 

INFORMATION TECHNOLOGIES 

27 

I. 

EXECUTIVE SUMMARY 

1.0 

Scope of the  Assessment:  Lessons  for the Fifth  Framework  Programme 

The  Panel's  five-year  assessment  of  the  European  Strategic  Programme  for  Research  in  Information 
Technologies  (ESPRIT)  is  based  on  an  examination  of  the  many  documentary  materials  supplied  by  the 
Commission,  including  the  1996 report  of  the  ESPRIT  Review  Board  produced  under  the  chairmanship  of 
Professor  Umberto  Colombo.  It  has  also  interviewed  senior  Commission  staff  concerned  with  Information 
and  Communications  Technology  (ICT)  policies.  In  addition,  the  Panel  sought  the  opinions  of  national 
delegates to ESPRIT. 

The  review  has  focused  on  activities  carried  out  under  the  European  Community's  Third  Framework 
Programme,  which  ran  from  1991  to  1994,  and  during  the  initial  period  of  the  Fourth  Framework 
Programme,  up  to  early  1996.  This  assessment  incorporates  the  final  evaluation  under  the  Third 
Framework  Programme. 

The  Panel  has  especially  focused  on  some  specific  features  of  the  present  ESPRIT  programme  and,  in 
agreement  with the  Commission,  on strategic  issues  affecting  ICT  research  and technology  development  in 
the  Fifth  Framework  Programme.  It  has  arrived  at  a  significant  number  of  constructive  recommendations 
that  reflect  the  Panel's  view  of the crucial  role  of ICT  and of the  importance  of the  Information  Society.  It 
hopes  that  the  Commission  will  endorse  this judgement  and  assign  a budget  to  the  future  ICT  programme 
in the Fifth  Framework  Programme which  is at least  equivalent  to the  present  one;  that  is,  about  one  third 
of the overall  Framework  Programme  Budget. 

2.0 

A Brief  History  of ESPRIT: Steadily  Widening  Objectives 

Community  funding  of  IT  R&D  begun  in  1984  .  During  the  first  decade,  the  programme  has  been 
essentially  aimed  at  closing  the  gap between  Europe's  information  technologies  industries  and  those  of  the 
US and Japan. 

The  strategy  changed  radically  with  the  launching  of  the  Fourth  Framework  Programme  in  1994  which 
recognised  the  increasingly  critical  role  played  by  IT  in  the  competitiveness  of  all  industry.  The  new  IT 
Programme: 

promotes  vertical  collaboration  between  ICT users and  suppliers; 

gives much  closer  attention to focusing  results on market  needs; 

facilitates  access to new technologies  and their take-up; 

encourages  an awareness  of emerging  R&D results; 

employs  a  number  of  new  tools  to  implement  the  programme  such  as  networks  of  excellence  and 
focused  clusters  (activities  covering  different  technological  areas with a single well-defined  goal). 

Council decision  84/30 (28.02.84) established  ESPRIT as part of the First Framework  Programme (1984 - 1988) 
with  a budget  of  ECU  1,500  million.  Council  decision  88/279  (11.04.88)  continued  ESPRIT  into  the  Second 
Framework  Programme with  a budget  of ECU 3,200  million  and Council  decision  91/2328 (15.07.91) renewed 
ESPRIT  as  part  of  the  Third  Framework  Programme  with  a  budget  of  ECU  2.035  million.  Council  decision 
94/802 (23/11/94) established a new programme, called Information  Technology Programme, as part of the Fourth 
Framework Programme with a budget of ECU 1,911 million. 

29 

3. 

3.0 

The Context: Europe's  Transition towards the Information  Society 

The Panel's  report  is written against the background  of rapid change in the importance,  development and 
use of ICT.  The moment is opportune to present some new ideas enabling Europe to adapt to at least four 
key developments: 

•  Europe's transition to the Information  Society has begun and is a crucial challenge for Europe's  future 

economy and society; 

• 

deregulation  of  European  telecommunications,  which should  be a major  factor  aiding  this transition, 
is now little more than a year away; 

• 

rapid growth in use of the Internet is bringing new opportunities to businesses and citizens; 

•  multimedia  technologies  are  penetrating  the  worlds  of  business,  the  media,  public  services  and 

education. 

This report urges radical  changes in the organisation  and design of ICT programmes  to enable the EU to 
respond better to these developments. 

4.0 

Priority  Issues: A Single  ICT Programme  Which Brings More Products to Market 

Two priorities have been identified  and addressed in the Panel's recommendations: 

4.1 

The need to merge the ACTS,  Telematics  and ESPRIT  programmes 

The  Panel  has  come  to the  conclusion  that  the  objectives  of  these  separate  programmes  would  be better 
achieved  if they  were brought  together  in a single programme  that covers  all  aspects of  ICT required by 
the Information  Society. 
Given  the  fact  that  ESPRIT  operates  on  the  basis  of  annual  work  programmes,  the  Panel  believes  that 
many of the changes in this direction can be implemented  on a step by step basis during  the remainder of 
the  Fourth  Framework  Programme.  This  will  help  to  smooth  the  transition  to the  Fifth  which  begins  in 
1999. 
An  attempt  to  define  a  timetable  for  the  programmes'  merger  would  have  been  outside  our  terms  of 
reference.  But we believe it would be helpful  to establish a joint coordination  committee between the three 
programmes  to  prepare  for  the  event.  We  are  aware  that  we  are  advocating  a  profound  change  which 
needs careful  planning and, above all, avoidance of administrative obstacles to what would be a significant 
improvement  in Europe's programmes for ICT. 

4.2 

The need to achieve much swifter  commercial  applications  of research  results 

The  panel  has  concluded  that  ESPRIT's  programme  objectives  have  been  satisfactorily  accomplished. 
Among  other  things,  it  has  made  a  vital  contribution  to  promoting  cross-border  collaboration  between 
small and large industries, universities and research institutes. 

However,  the programme could be even more effective  if - in addition to activities such as best practice, 
first-user  actions  and  assessments  -  better  mechanisms  were  established  for  ensuring  the  commercial 
exploitation of results. A number of actions could be envisaged, including: 

• 

• 

focusing  results much more explicitly on the tasks needed to construct Europe's Information  Society; 

initiatives to promote the commercial exploitation of research results; 

30 

3. 

• 

• 

• 

developing the tools to measure and monitor European ICT development, take-up and market shares; 

encouraging  SMEs  with  innovative  capacities  to  strengthen  and  speed  up  their  marketing  of  new 
value-added products; 

improving  the  efficiency  and  accountability  of  the programme's  administration,  partly  by  making  it 
more results-oriented; 

• 

define a long-term strategy for European research and technological development in ICT. 

5.0 

The Panel's Main  Findings 

The Panel's  findings  look back to the five  years of the assessment period  and forward  to the needs of the 
future  programme. 

5.1 

Assessment  Period  1991-1996 

5.1.1  Programme  Objectives: Time to Add The Needs of  Society  At Large 

The  Panel  agreed  that  ESPRIT's  objectives  have  been  relevant  and  fully  endorsed  its  new  focus  on 
responding to the needs of business and industry as a means of boosting competitiveness  and employment. 
However,  there is now an additional need to broaden the objectives to include the needs of society at large 
which  must  find  their  response  in  the  evolving  infrastructures,  services  and  applications  of  the 
Information  Society. 

Having  assessed  the programme's  performance  against  its objectives,  we would  draw  particular  attention 
to: 

• 

• 

• 

the  positive  change  in  the  dynamics  and  perspective  of  ESPRIT  from  the  Third  to  the  Fourth 
Framework Programmes; 

the  focus  on  relevant  topics  has been  made possible  by  a broad  description  of  research  tasks  and a 
continuously updated work programme; 

an  imbalance  between  the  user-need,  market-driven  priorities  and  the  requirement  for  long-term 
research.  The new focus is welcome but we are concerned that the necessary balance with long term-
research is being jeopardised. 

5.1.2  Programme  Efficiency:  Relevant Effort,  Still Some  Shortcomings 

The  Panel  considers  that  the  management  as  a  whole  has  made  great  efforts  to  raise  the  programme's 
performance  and  efficiency.  In  addition,  worthwhile  changes  have  been  introduced  to  make  the 
Programme more flexible  - a feature which must be characteristic of the Fifth  Framework Programme. 

However,  the  search  for  flexibility  has turned  on attempts  to  satisfy  the  specific  needs  of  proposers  and 
this,  paradoxically,  has made certain aspects of the programme very  complex (e.g.  variety  of actions and 
information packages, different  types of proposal submissions). 

The  Panel  also  wishes  to  draw  attention  to  the  more  general  problem  of  administrative  delays  which 
characterise  the  Framework  Programme  as a whole,  particularly  in concluding  contract  negotiations  and 
making payments. 

5.2 

The Future of ESPRIT: Five Challenges  for the Fifth  Framework  Programme 

The Panel has identified  five areas that must be addressed in the Fifth  Framework Programme: 

31 

3. 

(i) 

Programme  Strategy  and  Design 

(ii) 

Programme  management  and  evaluation 

(iii) 

The Information  Society 

(iv) 

Competitiveness  and  exploitation  of  research 

(v) 

SMEs and  start-up  companies 

5.2.1  Programme  Strategy  and  Design:  Three  Into  One  Will  Go 

Currently,  there  are  many  overlaps  between  ESPRIT,  ACTS  and  Telematics  which  need  to  be  turned  into 
synergies.  This  could  best  be  achieved  by  merging  these  three  ICT  programmes  into  one.  Within  this 
single  programme  a  proper  balance  must  be  struck  between  development  of  close-to-market  innovative 
applications  and  long-term  research. 

The  Panel's  detailed  recommendations  envisage  groupings  of  applications  and  technologies  within  one  ICT 
programme.  This  would  have  two  clear  purposes:  to  stimulate  innovative  applications  of  emerging 
technologies  and  to satisfy  the  requirement  for  strategic  and  long-term  research. 

5.2.2  Programme  Management  and  Evaluation:  In  Need  of  Overhaul 

Every  assessment  of  the  Framework  Programme  and  of  its  constituent  elements  as  well  as  a  survey  of 
national  opinion  undertaken  by  the  Panel  confirms  that  the  programme  management  has  become  more 
efficient.  Nevertheless,  the  Panel  supports  the  widespread  view  that  there  is  still  considerable  room  for 
improvement. 

Oversubscription  is  a  familiar  problem  with  Community-supported  initiatives  and  it  imposes  heavy  costs 
and  burdens  on  the  proposers.  It  is  important  to  resolve  the  difficulty  because  it  causes  frustration  and 
dissatisfaction  and  could  prompt  dynamic  companies  to abandon  their  interest.  The  limited  introduction  in 
ESPRIT  of  the  two-step  procedure  has  proved  an extremely  useful  attempt  to deal  with  the  situation  .  The 
Panel  recommends  extending  it to the whole  programme. 

The  Panel's  detailed  recommendations  seek  to make  the  future  ICT programme  an  efficient  instrument  for 
Europe's  ICT  community  in  which  delays  would  be  cut  by  a half,  and  project  performances  much  more 
strictly  evaluated. 

5.2.3  The Information  Society:  ICT as a Tool  for  Economic  Growth,  Social  Cohesion and 

Personal Development 

As the  Information  Society  evolves,  key  aspects  of  political,  economic  and  social  life  will  be  increasingly 
permeated  by  the  use  of  ICT  and  its  applications.  If  Europe  handles  the  transition  in  a  timely  fashion,  it 
will  significantly  strengthen  its  competitiveness  in  global  markets  and  cash  in  on  faster  economic  growth, 
more  employment  and a better  quality  of  life. 

By  contrast,  if  it continues  to  move  more  slowly  than  some  of  its competitors,  then  our  countries  will  find 
it very  difficult  to maintain,  let alone  increase competitiveness  across the broad  range  of their  economies. 

A  very  serious  obstacle  to  the  realisation  of  the  Information  Society  is  that  European  businesses  and 
individual  citizens  are  lagging  far  behind  the  US  and  some  other  countries  in  embracing  ICT  and  in 
exploiting  the  opportunities  it  offers  for  both  corporate  and  personal  growth.  The  need  to  integrate  the 
use  of  ICT  into  education  and  training  is paramount,  both  to  establish  computer  literacy  and  as  a  tool  for 
learning. 

32 

3. 

The Panel's detailed  recommendations  seek to ensure that the future  ICT programme  is a key pillar  in the 
construction  of  the  Information  Society.  It  should  support  projects  which  help  to  deliver  public  goods 
serving  citizens'  needs,  which  act as a catalyst  for  user-friendly  applications  and  which  foster  a growing 
market for the new technologies. 

5.2.4  Competitiveness  and  Exploitation  of  Research:  The  Former  Grows  if  the  Latter 

Succeeds 

Innovative  development  and  innovative  use  of  ICT has to be  both  visionary  and  focused  if  Europe  is to 
maintain and enhance its competitiveness  in the global market place. Competitiveness  creates employment 
and both can be secured  if Europe  nurtures  the development  and take-up  of  a world-class  ICT that both 
produces and uses ICT to offer  constantly  improved applications and services. 

Constant  improvement  would  result  from  a  better  commercial  exploitation  of  the  results  of  ESPRIT 
projects.  In  cases  where  an  attempt  is made,  windows  of  opportunity  are  frequently  missed  because  the 
process has been far  too slow to adjust  to rapid changes in the market place.  The Panel believes that the 
existing  eligibility  criteria,  combined  with  various  practices,  has  sometimes  led  to  the  creation  of 
somewhat  artificial  teams lacking clear commercial  objectives.  Relaxing some of the criteria  and  insisting 
on common commercial objectives would certainly improve the situation. 

The  Panel's  detailed  recommendations  aim  at  more  benchmarking  of  ICT-based  competitiveness  in  the 
future  programme,  measures  to ensure  a shorter  time-to-market,  encouragement  of  projects  which  have 
an entrepreneurial  final  objective and supporting measures to help commercial exploitation of results. 

5.2.5  SMEs  and Start-up  Companies: Finding the Right Responses to Particular  Needs. 

Small  and  medium-sized  enterprises  account  for  around  80%  of  Europe's  employment.  ESPRIT  is well 
able  to  provide  specific  support  to  SMEs  and  start-up  companies,  but  its  full  potential  is  not  being 
exploited. 

Even  though  the  SMEs  represent  30%  of  the  participants,  the  programme  would  benefit  from  more 
simplicity to encourage SMEs that have made serious efforts  to develop commercial ICT applications. Too 
few awards are made to SMEs to enable the sector to achieve the commercial opportunities open to it. 

The Panel's  detailed  recommendations  seek to endow the future  ICT  programme  with procedures  which 
are  more  user-friendly  for  SMEs,  structures  which  are  more  supportive,  funding  rules  which  are  more 
flexible  and special measures  for encouraging risk taking and for helping start-up companies. 

Recommendations 

All of  the following  recommendations  have been designed  to apply  to the  future  single  ICT programme, 
except for group A which concern the Framework Programme as a whole. 

A. 

Improving the  Framework 

A.l  Cutting  Delays 

a)  The  Commission  should  overhaul  its  current  administrative  functions  and  re-engineer  key 
procedures  in  order  to  cut  delays  by  half.  Payment  procedures  in  particular  must  be 
streamlined  to  reduce  the  time  from  submission  of  an  invoice  to  actual  payment  from  the 
current 90 days to 30 days. 

33 

3. 

b)  The  key  administrative  procedures  should  be  reviewed  to  speed  up  decision  making  by 
delegation  of authority  to appropriate  levels and by reduction  in the  layers  of decisions. 

c)  The  project  partners  should  be  allowed  to  charge  for  expenditures  from  the  day  of  the 
project's  approval  so  as  to  cover  their  costs  during  the  contract  negotiation  phase.  A  down 
payment  should  be made  immediately  upon contract  signature. 

d)  The  current  payment  scheme  based  on  statement  of  costs  should  be  abandoned  in  favour  of a 
deliverables-based  payment  scheme.  This  will  not  only  simplify  procedures  but  also 
encourage more  efficient  and productive  projects. 

e) 

to  raise  efficiency, 

In  order 
timeliness  and  effectiveness,  some  of  the  non-strategic 
management  tasks  undertaken  by  Commission  staff  should  be  wholly  or  partially  contracted 
out to competent  external  organisations. 

A.2  Effective  post-project evaluation. 

A  simple  method  must  be  developed  to  evaluate  projects  after  their  completion,  and  a  user's  guide  must 
describe  how  and  when  to apply  this method  to different  types  of projects.  Contractors  must  be  obliged  to 
provide  the  Commission  with the  necessary  information  to permit  effective  post-project  evaluation. 

B. 

Programme  Strategy  and  Design 

B.l 

Uniting  Strengths. 

a)  Separate  programmes  for  Telematics,  ACTS  and  ESPRIT  should  be  abandoned  and  be 
replaced  with  groupings  of  applications  and 
integrated  ICT 
programme.  This  is  what  is  necessary  to  bring  together  all  the  technologies  required  to  build 
the Information  Society. 

technologies  under  one 

b) 

In  allocating  funds  in  the  Fifth  Framework  Programme,  the  Panel  strongly  recommends  to 
assign  a budget  in line with  the  total  ICT programmes  in FPIV  and  not below  one  third  of  the 
overall  FPV  budget. 

B.2 

Facilitating  New  Ideas. 

a)  The  new  ICT  programme  must  emphasise  the  need  for  Europe  to become  more  innovative  by 
innovative  applications  using  emerging  and  enabling 

the  development  of 

stimulating 
technologies. 

b)  Exchanges  of  researchers,  corporate  engineers  and  professionals  from  hospitals,  schools, 
public  services,  etc.,  should  be  encouraged  to  create  a  better  understanding  between 
technology  inventors,  developers,  and users.  This  would  improve  the  rapid  application  of  new 
ICT  developments. 

B.3 

Incorporating Vision. 

a)  A  strategic  vision  of  the  future  organisation  and  functioning  of  Europe's  Information  Society 

must be constantly  developed  and  reviewed. 

b)  An  "ICT  Foresight  Board"  should  be  set  up  to  help  develop  a  common  European  vision  on 

long-term  research  and market  trends. 

34 

3. 

c)  Long-term  research  plays  an  important  part  in  the  shaping  of  Europe's  future  and  must  be 
given a secured  place  in European  research and technology  development  in ICT.  Around  15 
%  of  the  programme  budget  should  be  reserved  for  basic  long-term  research,  strategic 
research,  and radical innovation. 

d)  As the scope of the future  Framework Programme broadens to address the needs of society at 
large, the  European  Parliament  and  the Council  of Ministers  should  play  a more  active  role 
in  defining  strategic  goals,  evaluation  criteria  and  methods,  assessing  the  results  of 
programme evaluations, and steering the course of the programme. 

C. 

Programme Management and Evaluation 

C.l 

Serving the Client. 

a)  The future  ICT programme  must be a more efficient  instrument  to the  European  community 

of ICT users and suppliers. 

b)  To  reduce  the  overwhelming  volume  of  applications,  the  two-step  proposal,  introduced  in a 
limited  number  of areas of the programme,  should  be the norm  for  all  areas.  This should be 
based  on a short  2 -4 pages pre-proposal  which  would  be followed  by a full  proposal  only if 
the pre-proposal was accepted. 

c)  The Focused  Clusters,  introduced  for  the first  time  in ESPRIT under  FPIV,  should  be more 

widely used in the next ICT programme. 

C.2 

Increasing  Accountability. 

a)  The future  Framework Programme should set specific  goals for  European ICT development, 

take-up and market shares, and for a target growth in employment. 

b) 

Indicative  measures  of how the programme  is succeeding  in building  the Information  Society 
should  be established  to demonstrate  the effectiveness  of the programme  to  its stake-holders: 
the Member States, the regions and their citizens. 

D. 

The Information Society 

D.l.  Enabling the Information Society. 

a)  An  Information  Society  Cluster  should  be  organised  within  the  future  ICT  programme  for 
promoting  the  development  and  take-up  of  IS technology  in  such  areas  as  content  creation, 
information  management,  applications  for  citizens,  communities  and  companies.  The  Cluster 
should be allowed to receive ideas and project proposals on a continuous basis. 

b)  Proposals  should  be  given  a  priority  that  bring  together  ICT  suppliers  and  users  in joint 

efforts  to define,  develop, deliver and take up IS technology and services. 

c)  A  single  point  of  contact  for  Information  Society  projects  should  be  established  at  the 
Commission  with particular  responsibility  for  coordinating  the  allocation  of  ICT programme 
finance with the structural  funds. 

35 

3. 

D.2.  Networking Citizens and Communities. 

a)  Pilot  projects  should  be  identified  to  foster  understanding  and  acceptance  of  the  Information 
Society  and to encourage  the use of ICT in every  day  life.  Special  attention  should  be given to 
those  who  may  have  particular  difficulties  of  access  to  technology  or  in  using  it,  such  as  the 
elderly,  the unemployed  and the  handicapped. 

b)  Closer  links  aimed  at  promoting  the  use  and  understanding  of  ICT  must  be  established 
between  the  programme  and  authorities  responsible  for  education  and  training  in  Europe, 
typically  at the regional  level. 

c)  National  and 

regional  authorities 

together  with  non-governmental  and  professional 
organisations  (e.g.  teachers  and  doctors)  should  be  encouraged  to  submit  project  proposals, 
preferably  in  vertically  integrated  teams  involving  all  players  in  the  ICT  value  chain,  from 
suppliers to users. 

D. 3  Preparing the Citizen for the Information Society 

a)  Strong  encouragement  should  be given  to specific  projects  targeting  employment  issues,  such 
as  training,  retraining  and  life-long  learning  technologies  or  applications.  In  addition,  the 
likely  impact  of  a  proposed  project  on  employment  should  be  included  in  the  evaluation 
criteria. 

b)  The  future  ICT  programme  should  support  the  establishment  of  appropriate 

technical 
standards  not  only  to encourage  business  and  market  development  but  also  to protect  citizens' 
rights,  particularly  in the fields  of the multimedia  content  industry  and  electronic  commerce. 

c)  The  programme  shall  encourage  and  support  projects  that  specifically  address  the  issues  of 

low-cost  access to public  information. 

d)  The  future  ICT  programme  must  encourage  and  support  projects  aimed  at  improving  the  user 

interfaces  of  ICT applications  in general,  and  of training  systems  in particular. 

E. 

Competitiveness and Exploitation of ICT 

E.l.  Establishing Milestones 

a)  A  long-term  strategy  for  achieving  and  sustaining  European  competitiveness  through  the 
development  and  take-up  of  ICT  must  be developed  by  the  Commission  in  collaboration  with 
industry.  The  strategy  must  include  clear  objectives,  market-share  targets,  and  the  expected 
contribution  from  RTD.  A  unit  should  be  set  up  to  monitor  how  Europe  is  progressing 
towards  its goals. 

E.2 

Shortening Time to Global Markets. 

a)  The programme  must  encourage  industry  to develop and  bring  ICT  applications  to the  market 

in shorter  time. 

b) 

It  must  emphasise  collaborative  efforts  in  commercial  exploitation  and  facilitate  a  market-
driven  approach  by  shortening  the  life  of  projects  so  that  time-to-market  periods  are  between 
1-2  years. 

36 

3. 

E.3 

Selecting the Fittest. 

a)  The  programme  must  support  those  who  can  and  will  exploit  the  results  of  European  funded 
R&D,  and discourage  those partners  who have no real  interest  or  capacity  for  exploitation. 

b)  More  efficient  models  of  cooperation  should  be  encouraged,  with  smaller  partnerships  built 
around  real  interests.  Every  partner  in  a  team  must  have  a  distinctive  and  necessary  role.  In 
evaluating  proposals,  the  team  efficiency  and  the  potential  for  commercial  exploitation  of  the 
project  results must be given higher  weighting. 

E.4  Moving Exploitation to the Top of the Agenda. 

a)  The  Commission  should  set  up  a  Help  Desk  including  a  network  of  experts  in  commercial 
applications,  marketing,  industrial  and  trade  law,  banking  and  venture  capital  to  provide 
coordinated  support  for  commercial  exploitation  and to encourage  benchmarking. 

b)  An  Innovation  Financing  Help  Desk  should  be  set  up  and  linked  to  the  Framework 

Programme  as proposed  in the Commission's  Action  Plan for  Innovation. 

c)  The  Commission  should  foster  the  creation  of  an  advisory  panel  involving  major  players  with 
a  decisive  influence  on  the development  of  an  effective  single  European  ICT  market  (utilities, 
PTTs,  regulators  etc.). 

d)  The  Commission  should  establish  and  fund  Reference  Sites  where  European  applications  can 
be  demonstrated.  Such  demonstrations  must  be  well  publicised  and  accessible  via  the  WWW. 
Sites  to  facilitate  local  access  should  be  established  in  large  potential  markets  in  the  US, 
Japan  and the  Far  East. 

e)  Priority  must  be  given  to  achieving  a  more  rapid  penetration  of  world-wide  markets  by 
helping  companies  to  set  up  subsidiaries  and  joint  ventures  in  developed  markets.  These 
should  be  treated  as  start-up  companies  for  the  purpose  of  RTD  in  the  new  Framework 
Programme,  provided  they bring  employment  to Europe. 

F. 

SMEs and Start-up Companies 

F.l  Making Life Easier. 

a)  The  Commission  should  establish  an  SME  Desk  as  a  single  point  of  contact  between  SMEs 
and  the  programme.  The  purpose  of  the  Desk  should  be  to  provide  full  support  and  to 
eliminate  the  present  difficulties  companies  have  in  navigating  through  the  programmes.  The 
Desk  would  evaluate  ideas  for  proposals  and provide  all  of  the  necessary  information  on  what 
is required  of a proposer  and  when. 

b)  Create  a  new  category  of  Small  and  Medium-sized  Project  (SMP)  aimed  mainly,  but  not 
exclusively,  at  SMEs  which  runs  for  less  than  one  year  and  with  Commission  funding  not 
exceeding  ECU  500,000 per  project. 

c)  Fast-track  support  should  be  established  for  Small  and  Medium-sized  Projects,  allowing 
the  SME 

adoption  of  unsolicited  proposals  not  linked  to  a  particular  call.  The  services  of 
Desk  would be at the disposal  of  SMEs wishing to pursue  a small  project. 

d)  Some  future  ICT  programme  funding  -  around  5%  -  should  be  earmarked  for  special  calls 
aimed  at stimulating  partnerships  in which  SMEs take the  leading  role  in the  development  and 

37 

exploitation  of  results.  The  adequacy  of  the  funding  should  be  judged  in  the  light  of 
scheme's  success  in creating  these highly  desirable  partnerships. 

3. 

the 

e)  Efforts  should  be  made  to  thoroughly  familiarise  SMEs  with  the  future  ICT  programme  and 
its objectives  by  involving  them  more  closely  in the  consultation  process  and  by  extensive  use 
of external  evaluators  and project  reviewers  drawn  from  SMEs. 

F.2  Matching  Market  Endorsement. 

a)  Programme  funding  must be made more flexible  in order  to provide  specific  incentives to 

SMEs with exploitable  ideas. 

b)  Funding  should  be  increased  to 60%  when  the potential  commercial  validity  of  an  exploitable 
project  has been  endorsed  by  venture  capital  backing,  or by  a large company  covering  at  least 
10%  of  a project's  costs without  itself  participating. 

F.3 

Stimulating  Start-Ups. 

a)  Maximum  funding  of  75  % should  be  available  for  projects  with  good  exploitation  prospects 

involving  start-up  companies  which  are not subsidiaries  of other  companies. 

b)  Grants  should  be  available  to  start-up  companies  which  are  partners  in  successful  projects  to 
cover  the  costs  of  finding  and  securing  investment  capital  for  commercial  exploitation.  These 
could  be  10 -  15  % of the amount  that the  EU contributed  to the start-up's  part  of the  project. 

38 

IL  COMMISSION'S RESPONSE 

The  independent  evaluation  of  the  programme  over  the  last  five  years  was  prepared  by  the  high  level 
group  of  experts  chaired  by  Professor  Roberto  Carneiro.  The  evaluation  was  based  on  the  examination  of 
documentary  material  provided  by  the  Commission  including  the  final  report  produced  under  the 
chairmanship  of  Professor  Umberto  Colombo  which  evaluated  the  activities  supported  by  the  programme 
under  the  previous  Framework.  The  Commission  welcomes  the  recommendations  of  the  panel  which 
essentially  address the future  orientations  of the programme. 

The  Commission  views  the  main  findings  and  recommendations  as  generally  useful  for  the  remaining 
period  of  the  current  Framework,  and  for  acting  as  guidelines  for  the  current  discussions  associated  with 
the  preparation  of  the  5th  Framework  Programme.  The  Commission  welcomes  the  recommendation  that  a 
number  of  new  approaches  introduced  in  the  current  programme  (e.g.  rolling  workplan,  regular  calls, 
two-step  evaluation)  be generalized  as appropriate  in the  Fifth  Framework  Programme. 

The  need  for  combining  various  technologies  and  actions  towards  the  establishment  of  an  "user-friendly 
Information  Society"  has  been  stressed  in  the  Communication  "Inventing  Tomorrow".  The  Commission 
welcomes  the  panel's  views  on  this  topic  and  shares  its  conviction  that  greater  convergence  of  efforts  is 
necessary  and  that  a  single  and  integrated  programme  addressing  Information  and  Communication 
Technologies  is  required.  As  suggested  in  the  executive  summary,  the  programme  will  take  advantage  of 
its  rolling  workprogramme  to  start  implementing  changes  during  the  Fourth  Framework  Programme  to 
smooth  the transition  into the  fifth. 

The  Commission  supports  in  particular  the  view  that  technology  take-up  should  be  further  emphasised  in 
the  forthcoming  programme  to  imrove  competitiveness  and  create  employment  through  a  more  wide 
spread  exploitation  of  ICT.  The  Commission  welcomes  the  panel's  emphasis  on  the  role  of  citizens  and 
communities.  Technological  progress  in  the  ICT  area  increasingly  depends  on  dynamic  markets  for  new 
products  and  services  and  hence  on  the  ability  and  willingness  of  citizens  and  workers  to  use  new  ICT. 
The  balance  between  long-term  research  and  take-up  activities  needs  to  be  defined  according  to  the 
industrial  and  socio-economic  objectives  of  the  forthcoming  programme.  A  strategic  and  forward-looking 
vision  of  research  and  social  developments  is  certainly  required,  as  mentioned  in  the  "action  plan  for 
innovation"  to  identify  promising  technologies  and markets. 

The  Commission  welcomes  the  recommendations  aiming  at  improving  SMEs'  access  to  the  programme. 
The  suggestions  made  by  the  panel  support  the  "Inventing  Tomorrow"  proposal  to  help  SMEs  access  the 
programme  by  developing  technology  transfer  mechanisms  and  by  establishing  links  with  venture  capital 
and  financial  engineering  tools. 

Finally,  the  Commission  supports  the  view  that,  at  the  Framework  level,  additional  efforts  need  to  be 
made  to  improve  the  flexibility  associated  with,  and  the  efficiency  of,  administrative  procedures,  to 
shorten  the  deadlines  and  administrative  costs  so that  dynamic  companies,  in particular  SMEs,  are  further 
encouraged  to join  the future  programme. 

39 

TÏT.  EVALUATION  PANEL 

Prof.  Roberto  Carneiro  (Chairman),  President  of  TVI,  Televisào  Independent  S.A. 
Shareholders)  Group  Forum,  SGPS. 
Former  Minister  of  Education  and  Sports  in  Portugal  (1987-91),  senior  World  Bank  (IBRD  and  IDA) 
consultant  for  projects  in Latin  America  and  Africa  (1975-87);  and  since  1978 OECD  senior  consultant  on 
education  policy,  manpower  planning  and  public  management;  and  UNESC  senior  consultant  and  expert 
on  education  and  training  policy.  Recent  international  activities  include  consultant  coordinator  to  SIGMA, 
EEC/OECD joint  venture  designed  to  implement  State  reform  in  Central  and  Eastern  European  Countries 
(1992-93);  OECD  Examiner  of  the  French  Educational  Policy  (1993);  Member  of  the  International 
Commission  for  Education  in  the  21st  Century  (UNESCO,  93/95);  vice-president  of  the  Information 
Society  Forum  and  Member  of  the  Bureau  which  coordinates  the  European  Commission  Study  Group  on 
Education  and  Training  under  the auspices  of Commissioner  E.  Cresson. 

(Board  of 

Andrew  Bos well,  Managing  Director,  Enterprise  Technology  and  ICL  Technical  Director. 
Chairman  of  the  UK  National  Information  Infrastructure  Task  Force,  which  works  with  the  Department  of 
Trade  and  Industry  on  ICT  related  issues.  Appointed  to  the  G7  Committee  responsible  for  accelerating 
standardisation  in ICT,  an  initiative  that stemmed  from  the G7 summit of  1995. 

Jean-Michel  Chassériaux,  Managing  Director  ofERCIM  and  Director  of  International  Relations 
atlNRIA. 
Scientific  attaché  to  the  French  Embassy  in  London  (1973-77),  senior  consultant  for  the  Ministry  of 
Foreign  affairs  and  the  Ministry  of  Industry  (1977-84).  Deputy-director  for  training,  research  and  regional 
affairs  at  the  ICT  industries'  Unit  of  the  Ministry  for  Industry  (1984-87).  Director  for  international  affairs 
at the Ministry  of  Research  and  Space (1988-93).  Professor  at the Paris  7 University  since  1983. 

Bernt  Ericson,  Vice-President,  Research  and  Technology,  Telefonaktiebolaget  LM  Ericsson. 
Joined  Ericsson  in  1969  as  Chief  Engineer  and  member  of  the  Business  Strategy  Group;  Department 
Manager  for  special  projects  within  the  Strategic  Product  Planning  division;  Director  of  applied  Research 
and  finally,  vice-president.  Elected  as  member  of  the  Royal  Swedish  Academy  of  Engineering  Sciences 
and  of the  European  Science  and  Technology  Association(ESTA) of the  European  Commission. 

Roberto  Galimberti,  Chairman  and  CEO  of  Etnoteam. 
From  1968  to  1973  head  of  Digital  Computing  Division  of  Laben.  With  Italtel  as  Research  and 
Development  Director  and  later  as head  of the Toll  Switching  Division.  Since  1982 Chairman  and  CEO of 
Etnoteam.  Since  1994 Chairman  and  CEO ofl.NET,  a leader  in Internet  business  connectivity.  Also vice-
president  of  CED  Borsa  (Italian  Stock  Exchange  Information  System  Company),  and  of  Intercai  Etnoteam. 
Member  of the board  of Anasin  (Italian  Computer  Services  Association). 

Hartmut  Raffler,  Senior  Director,  Siemens  Corporate  Technology  Division. 
Joined  Siemens  AG  in  1979.  Responsible  for  the  Innovation  Project  Communications.  From  1993 to  1996 
Head  of  the  Software  and  Engineering  Department.  Since  April  1996  Head  of  the  Information  and 
Communications  Department.  Main  topics:  Information  and  Media  Technologies,  Networks  and  Video 
Communications,  Security  in Systems, Knowledge  Processing  and  Man-Machine  Cooperation. 

Josep  Maria  Vila  Solanes,  General  Manager  of  INDRA  Group,  President  of  INDRA  SSI  and 
President  of  INDRA  SCA. 
Chief  Executive  Officer  of  ERITEL,  General  Manager  of  Olympic  Games  Organising  Committee  at 
'92.  Deputy  Manager  of  ENHER  and  Branch  Manager  of  UNISYS.  Director  of  the 
Barcelona 
DATAMATION  magazine 
the  Nuclear  Technology  Department, 
Universidad  Politécnica  de  Barcelona,  and  formerly  Professor  in other  universities  and  research  centres  in 
Barcelona. 

(Spanish  edition).  Professor 

in 

40 

Herbert  Wotke, Managing  Director  of the Austrian  Industrial Research  Promotion Fund (FFF). 
Served for more than 2 decades in the FFF (Forschungs Fôrderungs Fonds fur  die gewerbliche  Wirtschaft) 
in  national  and  international  funding  activities  including  technology  policy,  EUREKA  and  the 
governmental  "Innovation and Technology  Fund". 

3. 

41 

1996 FIVE-YEAR ASSESSMENT 

INDUSTRIAL MATERIALS AND TECHNOLOGIES 

43 

I. 

EXECUTIVE  SUMMARY 

1.0 

Evolution  of  the  Industrial  and  Materials  Technology  Programmes 

the  First  Framework  Programme.  Brite  was  concerned  with  a  wide  range  of 

Among  European  decision-makers,  there  is agreement  that  industrial  and  materials  technologies  are  of  key 
importance  to  the  future  success  of  European  manufacturing  industry.  The  present  IMT  programme, 
alternatively  known  as  'Brite-Euram',  stems  from  two  RTD  programmes,  'Brite'  and  'Euram',  conducted 
under 
industrial 
manufacturing  technologies,  while  Euram  was  devoted  to  advanced  materials.  In  the  Second  Framework 
Programme(FP2),  the  two  subject  areas  were  brought  together  in  'Brite-Euram',  and  work  related  to 
aeronautics  was  added.  This  arrangement  has  continued,  with  the  further  addition  of  surface  transport 
technologies  in the current  Fourth  Framework  Programme. 

The  central  emphasis  of  the  work  under  these  programmes  has  been  on  precompetitive,  generic  research, 
conducted  in  a  collaborative  manner  across  national  borders  and  involving  industry,  research  institutions 
and  universities.  Throughout,  an  aim  has  been  to  strengthen  the  scientific  and  technological  basis  of 
European  industry  and  encourage  it to become  more  competitive  at  international  level.  Particular  attention 
has been paid  to  the  support  and  encouragement  of  SMEs,  and  to  the  development  of  European  economic 
and social  cohesion. 
Successive  IMT  programmes  have  been  expanded  in  funding  and  scope.  FP4  IMT  is  considerably  larger 
than  FP3,  and  has  a broader  range  of  objectives  including  improving  the  quality  of  life  while  promoting 
sustainable  economic  growth.  IMT funding  is summarised  below. 

FP1 
220 MECU 
(Brite  +  Euram) 

FP2 
500 MECU 
(Brite-Euram) 

FP3 
748 MECU 
(Brite-Euram  II) 

FP4 
1617 MECU 
(Brite-Euram III) 

2.0 

Scope of this Five-Year Assessment 

This  Five-Year  Assessment  covers  the  Industrial  and  Materials  Programmes  carried  out  under  the  Third 
Framework  Programme( 1991-94) and  in the first  year  of implementation  of the  Fourth 
Framework  Programme( 1994-98) 

In  accordance  with  the  Terms  of  Reference,  the  assessment  evaluates  three  principal  aspects  of  the 
programmes,  namely;  relevance  of  the  objectives:  the  efficiency  with  which  the  programmes  have  been 
conducted;  and  the  effectiveness  of  the  programmes.  On  the  basis  of  these  evaluations,  recommendations 
are made  regarding  future  programmes 

This  Executive  Summary  forms  the  first  part  of  'Document  1',  which  also  contains  the  Commission 
responses  to the Panel's  recommendations.  The Panel's  full  Report  is presented  in  'Document  2'. 

This assessment  incorporates  the final  evaluation under  the Third  Framework  Programme. 
At  the  same  time,  however,  the  Panel  wishes  to  draw  attention  to  the  two  Post-Evaluation  Reports 
covering  Areas  1&2  (EUR  17457)  and  Area  3  (EUR  16948)  of  this  programme.  These  Post-Evaluations 
were  carried  out  in  depth,  by  individuals  knowledgeable  and  experienced  in  the  fields  concerned,  and 
produced  a range  of  specific  Conclusions  and Recommendations  which  call for  attention.  In taking  its own 
overview,  the  present  Panel  aims  to  add  a  useful  general  perspective  but  this  should  not  be  regarded  as 
repacing  or  detracting  from  the  important  content  of the Post-Evaluation  Reports. 

3.0 

The Panel's Comments and Findings 

3.1 

Relevance of Obj ectives 

A  wide  range  of  objectives  has  been  set  for  the  IMT  programmes;  wider  than  for  typical  industrial  or 
nationally-run  programmes.  The  Panel  accepts  the  general  validity  and  continuing  relevance  of  the 

45 

4. 

objectives,  but  recognises  the  difficulty  of  satisfying  all of them  simultaneously  at project  level.  In view  of 
the  quickening  pace  of  globalisation  in  trade,  services,  and  manufacturing,  and  the  need  to  strengthen 
European  cohesion,  optimal  operation  of  the programme  requires prioritisation  of  objectives,  and  hence of 
project  selection  criteria,  to avoid  resources  being  spread  too thinly  and  focus  reduced. 

A  more  suitable  format  for  the  IMT  programme  in  FP5  would  be  a  combination  of  'horizontal'  generic 
work  with  a  number  of  selected  areas  where  a more  focussed  'vertical'  approach  is pursued.  These  focus 
areas  might  relate  to the  Task  Force  interests,  though  others  should  also be  considered  if  appropriate.  The 
modalities  of  this  programme  concept, 
implementation  of  additional  RTD 
instruments,  need  investigation. 

including  the  possible 

Selection  criteria  such  as  scientific  and  technical  excellence,  economic  and  social  cohesion,  sustainable 
development  and  quality  of  life,  generic  and  precompetitive  research,  transnational  participation,  effective 
participation  of  SMEs,  and  exploitation  capability  have been generally  adhered  to. 

3.2 

Assessment of  Efficiency 

The  issue  of  cost-effectiveness  as  a  measure  of  the  efficiency  with  which  resources  have  been  used  in 
pursuit  of  the  programme  objectives,  relates  to  both  Commission  resources  and  programme  participants' 
resources. 

The  financial  contributions  made by  the Commission  have  been  generally  cost-effective.  The  funding  rules 
have  provided  an  appropriate  incentive  for  participation,  while  at  the  same  time  requiring  sufficient 
investment  by participants  to ensure  serious  commitment. 

Cost-effectiveness  regarding  participants'  resources  depends  on  the  balance  between  the  advantages  of 
working  collaboratively,  with  a  Commission  funding  contribution,  and  the  extra  management  effort 
needed.  Generally  this  balance  is  positive,  but  preparation  of  proposals  requires  considerable  effort, 
averaging  about  5%  of  project  cost  for  each  proposal.  There  is  also  a  high  proportion  of  unselected 
proposals,  although  indications  are  that  in  nearly  half  of  these  the  research  would  be  done  anyway.  To 
reduce  the  total  resource  cost  of  proposals,  and  to  avoid  deterring  capable  potential  participants  from 
applying,  the  aim  should  be  to  improve  the  success  rate  to  more  than  50%.  A  two-stage  procedure,  in 
which  unattractive  proposals  are  rejected  on  the  basis  of  a  less-detailed  first-stage  submission,  should  be 
considered. 

The  efficiency  of  programme  management  relates  to activities  in advance  of  the  start  of  project  work  as 
well  as to  management  of  the  work  as  it progresses.  A continuing  concern,  expressed  both  by  programme 
participants  and  previous  Independent  Panels,  is the  long  timescale  between  close  of  call  for  proposals  and 
start  of  work.  For  both  FP3  and  FP4  IMT  programmes,  this  ranged  between  9  and  15  months.  This 
compares  poorly  with  nationally-run  programmes,  including  those  of  important  competitor  nations  such  as 
the  USA.  This  timescale  should  be  reduced  to  not  more  than  6  months.  Mechanisms  to  allow  work  to  be 
started  with  limited  contract  cover  prior  to final  contract  signing  should  be  investigated. 

Management  and  monitoring  of  the  work  under  contract  has  been  generally  satisfactory,  both  by  the 
Commission  and  within  the project  consortia.  Positive  comments  have been  received  regarding  the  quality, 
competence  and  dedication  of  the  Commission  Scientific  Officers.  However,  there  is  evidence  that,  with 
the  expansion  of the  IMT programme,  the  greater  number  of  Member  States  and  the  increased  complexity 
is  being 
of  many  proposals,  Commission  staff  are  under  increasing  strain.  Programme  efficiency 
endangered  and  a review  of organisation  and procedures  is needed. 

The  use  of  external  experts  to  advise  and  assist  Commission  staff  can  benefit  the  efficiency  and 
effectiveness  of  project  management  -  for  example,  by  providing  an  independent  view  ,  and  a  healthy 
stimulus,  at  critical  project  reviews.  However,  the  Panel  wishes  to  emphasise  that  for  efficiency  and 
continuity  reasons  it  is  essential  that  an  adequate  core  of  technically  competent  personnel  should  be 
maintained  within the  Commission. 

46 

4. 

The  Programme  Committees,  forming  the  interface  at  programme  level  between  the  Commission  and 
Member  States,  have  played  a key  role  in the  planning,  formulation  and  implementation  processes.  When 
specific  funds  are  allocated  to  a  particular  area,  eg  aeronautics,  it  is  appropriate  to  involve  a  panel  of 
individuals  with  specialist  knowledge  to  ensure  that  the  programme  continues  to  develop  according  to 
market  needs. 

The Task  Forces  have so far  had  little  influence  on the IMT Programme.  They  did  not come  into  existence 
until  March  1995, by  Which time  FP3  was  in its closing  stages,  and  FP4  had  been  formulated  and  the  first 
call  for  proposals  launched. 

In  regard  to  external  monitoring,  the  well-established  process  of  'Post-Evaluation'  provides  an  overall, 
in-depth  perspective  on  the  whole  course  of  a programme,  based  on  documentation,  interviews,  visits  and 
questionnaires.  This  leads  to  valuable  comments  and  recommendations  for  the  future,  and  should  be 
continued. 

A  useful  complement  to  Post-Evaluation  is  the  newly  instituted  'Annual  External  Monitoring',  which 
provides  an  annual  check  on  progress,  thus  forming  a  basis  for  deciding  on  adjustments  of  emphasis, 
procedures,  etc.  However,  the the value  of the process  would  be enhanced  if that  panel  had  some  access  to 
the  views  of  those  involved,  perhaps  by  means  of  a one-day  'workshop'  with  national  representatives  and 
project  participants. 

In  relation  to  modes  of  action  and  tools,  while  there  is  certainly  an  important  continuing  role  for 
precompetitive.  generic  research,  the  Panel  believes  that  in  view  of  the  increasing  intensity  of  global 
competition,  shortening  of  time-to-market,  etc,  the  Commission  should  now  consider  complementing  this 
with  nearer-to-market  RTD  but  still  stopping  short  of  product  development.  In  addition,  consideration 
should  also  be  given  to  RTD  invoking  Treaty  Articles  130  K.L  and  M  and,  in  selected  cases,  adding  a 
proportion  of 
'directed'  research,  strategically  planned  by  or  on  behalf  of  the  Commission,  as  a 
complement  to the  established  style of  'invited'  research. 

Basic  Research  projects  of  industrial  relevance,  involving  research  institutions  and  universities  working  in 
conjuction  with  industrial  enterprises,  have  proved  to  be  a  very  valuable  aspect  of  RTD  under  the  IMT 
programme  and  should  be  continued. 

The  significance  of  SMEs  within  the  European  industrial  scene  is well  recognised  by  the  Commission  and 
is fully  endorsed  by  the  Panel.  In  the  IMT  programmes,  support  and  encouragement  of  SMEs  has  been 
shown  by  the  extensive  involvement  of  already  technologically-capable  SMEs  in  industrial  RTD  projects, 
and  by  the  introduction  and  development  of  the  CRAFT  scheme  which  supports  'cooperative'  research 
commissioned  by groups  of  SMEs which  currently  lack an internal  RTD  capablity. 

The  participation  of  technologically-capable  SMEs  in  industrial  RTD  projects  has  increased  in  recent 
years.  The  involvement  with  larger  companies  and  with  academia  across  national  boundaries  has  provided 
SMEs  with  access  to new  technology  options  and  new  commercial  opportunities.  On  the  other  hand,  there 
is evidence  that  some  SMEs  are  deterred  from  becoming  involved  in project  proposals  because  of  the  long 
timescales  and  low  success  rate,  or  because  they  fear  a  possible  restriction  of  their  flexibility  during  the 
course  of  a  project.  SMEs  are  often  subject  to  more  dynamic,  fast-changing  circumstances  than  large 
organisations. 

The  CRAFT  scheme  has  proved  to  be  an  excellent  means  for  assisting  SMEs  with  a  relatively  low 
technology  base,  to  raise  their  technical  capabilities.  The  two-step  evaluation  process  is  very  successful. 
The  Panel  regards  CRAFT  as  a high-priority  action,  and  welcomes  the  higher  funding  allocation  in  FP4. 
The  CRAFT  action  is however  demanding  in terms of Commission  management  effort,  and  more  attention 
to the  staffing  requirement  is needed  for  the future  development  of CRAFT.  Consideration  should  be  given 
to  a  degree  of  decentralisation  of  stimulation  to  Member  States,  and  the  'bottom-up'  approach  should  be 
maintained. 

47 

4. 

The  Panel  also  wishes  to  emphasise  that  the  interests  and  needs  of  SMEs  need  to  be  kept  in  balance  with 
those  of  larger  industrial  firms,  as both  large  firms  and  SMEs have  important.contributions  to make  to the 
EU. 

In  regard  to  Coordination  Mechanisms,  both  the  'Thematic  Networks'  and  the  Cluster  Concepts'  of  FP4 
are  still  in  a  relatively  early  state.  For  fuller  exploitation  of  their  potential  it  will  be  necessary  to  go 
beyond  merely  the  coordination  of  RTD  items  which  arise  separately  and  independently  in  various 
Commission  programme  areas.  A more  proactive,  strategic  approach  will  be  needed,  to  set  up  integrated, 
multidisciplinary  groups  of  projects  with  forward  planning  to  ensure  coverage  of  all  aspects  essential  for 
commercial  exploitation  in a competitive  timescale. 

3.3 

Assessment  of  Effectiveness 

The  Post-Evaluation  Reports  for  FP3  IMT  indicate  that  the  initial  objectives  of  promoting  technical 
research  in  all  Member  States,  supporting  and  encouraging  SMEs  and  the  collaboration  of  industry, 
research  institutions  and  universities  across  national  boundaries,  were  met  to  a  high  degree.  The  Reports 
also  judge  positively  the  achievement  of  project  and  programme  objectives,  including  contributing  to 
European  cohesion  and  economic  development.  The  number  of  patents  filed  averaged  less  than  one  per 
project,  but patents  are  not  a universal  measure  of  achievement.  Much  important  innovation  within  IMT  is 
not  patentable,  eg  advances  in  scientific  disciplines  such  as  structural  analysis,  fluid  mechanics,  etc.  The 
considerably  larger  FP4  Programme  is  still  in  its  early  stages  and  therefore  difficult  to judge,  but  strong 
responses  in most  programme  areas  are  apparent. 

The  overall  objectives  of  the  IMT  Programmes  during  the  past  five  years  have  been  met  in  regard  to 
scientific  and  technical  accomplishments.  An  open  question  is  whether  the  resulting  contributions  have 
been  adequate  to  improve  European  competitiveness,  in  view  of  the  increasingly  strong  RTD  activities  in 
competitor  countries. 

The  strengthening  of  the  scientific  and  technological  bases  has  benefitted  SMEs,  large  companies, 
universities  and  research  institutions  throughout  the  EU.  In  addition  to  the  accumulation  of  research 
results,  it contributes  to  the  forming  of  collaborative  relationships  after  project  completion  and,  especially 
for  SME  participants,  to  the  building  of  confidence  for  operating  at  higher  technological  levels.  In  this 
respect,  participation  in  the  CRAFT  scheme  has  brought  particular  benefits  to  SMEs  lacking  an  internal 
RTD  capability. 

It  is possible  to  identify  considerable  contributions  to  relevant  Community  policies  through  the  support  of 
cooperation  across  national  borders,  and  the  collaboration  of  SMEs,  research  institutions  and  universities. 
Economic  and  social  cohesion  has  been  strengthened.  There  has  been  a  small  direct  contribution  to 
employment,  by  recruitment  of  individuals  to  work  on  projects,  and  a  potential  created  for  larger 
employment  benefits  through  industrial  exploitation  in  future.  Also,  significant  scientific/technical  results 
have  been  obtained  in  areas  relating  to  the  environment  (recycling,  less  wasteful  manufacture,  more 
environmentally-friendly  materials,  aircraft  noise and  exhaust emissions,etc) 

In  the  areas  of  dissemination  and  exploitation,  a  balance  needs  to  be  achieved  between  commercial 
exploitation  solely  by the project  partners  who have invested  in the work,  and the distribution  of  results  for 
wider  benefits  to  the  EU.  FP3  IMT  results  have  been  given  controlled  dissemination  through  numerous 
Information  Days,  Workshops  and  Conferences.  Despite  these  significant  efforts,  however,  there  remains 
a  concern  that  many  potential  European  beneficiaries  of  the  results  of  the  IMT  programmes  are  not  yet 
being  reached,  and  more  attention  needs to be paid by the Commission  to dissemination  and  exploitation. 

Quantitative  estimates  of  exploitation  potential  in  reports  produced  under  Commission  contract  have 
indicated  very  high  economic  returns  on  programme  investments.  Comparisons  should  be  made  between 
such  estimates  and  the  actual  benefits  achieved,  by  re-visiting  a  sample  of  projects  3  to  5  years  after 
project  completion. 

48 

4. 

3.4 

Importance  of the IMT  Programme 

The  IMT  programme  is  an  important  and  excellent  programme.  Although  the  IMT  budget  in  FP4  is 
considerably  larger  than  for  FP3,  it  still  represents  a  small  proportion  of  the  total  requirement  for  RTD 
expenditure  in  this  field  in  Europe.  The  high  application  rate  of  good-quality  industrial  research  projects 
demonstrates  the  strong  willingness  of  European  industry  and  the  research  community  to  participate,  and 
points  to  the  need  for  IMT  funding  to  be  further  expanded  to  enable  the  Commission  to  respond 
adequately  to  this  willingness,  and  to  the  challenge  of  strengthening  the  technological  competitiveness  of 
European  industry. 

4.0 

Recommendations 

1.  The  IMT  Programme  is  considered  to  be  an  important  and  excellent  programme.  It  should  be 
significantly  enhanced  within the  Framework  Programme  and very  high  importance  should  be  allocated 
to  the  objective  of  strengthening  the  technological  competitiveness  of  European  industries.  The 
financial  means  and  personnel  available  for  the  IMT  programme  are  modest  for  the  task  in  hand  and 
should be  substantially  expanded. 

2.  The  IMT  programme  should  in  future  combine  a basis  of  'horizontal'  generic  work  with  a  number  of 

selected  areas where  a more  focussed  'vertical'  approach  is pursued. 

3.  Measures  should  be taken to increase the success rate of proposals  to higher than 50%.  The present  low 
success  rate  is  considered  to  discourage  prospective  participants  from  submitting  proposals  potentially 
important  to future  European  competitiveness. 

4.  In  order  to  reduce  the  cost  and  staff  effort  associated  with  proposal  preparation,  and  the  length  of  the 

processing  steps,  a two-stage  submission procedure  is  recommended. 

5.  To  make  the  IMT  programme  more  efficient  and  effective,  the  timescale  between  proposal  submission 

and the start of funding  of the work  should  not exceed  6 months. 

6.  SMEs  qualified 

to  participate 

in  IMT  programmes  should  be  encouraged 

to  apply.  Better 

information  and communication  with SMEs through  expanded  networks  and other means  is necessary. 

7.  CRAFT  is  an  excellent  action  in  IMT  and  should  receive  continued  emphasis.  Commission  staffing 
levels  in  CRAFT  should  be  increased.  Partial  decentralisation  to  Member  States  of  the  stimulation  of 
CRAFT  involvement  should  also be  considered. 

8.  Better  measures  for  dissemination  of  results  of  IMT  projects  should  be  introduced,  together  with  an 

increased  focus  on the encouragement  of  exploitation. 

9.  Sample  groups  of  IMT  projects  should  be  re-evaluated,  3 to  5  years  after  completion,  to  compare  the 

economic  returns actually  achieved  with the estimates  of exploitation potential  carried  out  earlier. 

10.To  encourage  the  beneficial  spin-off  to  other  industrial  sectors  of  IMT  project  results  aimed  firstly  at 
aeronautical  applications,  it  is  recommended  that  a  special  information/advisory  unit  be  set  up  to 
promote  wider  awareness  of such advances. 

11.Continued  emphasis  on  the  use  of  universities  and  research  institution  expertise,  especially  in  Basic 
Research,  is  recommended.  Basic  Research  in  support  of  Industrial  technologies  is  considered  very 
important  in the context  of the IMT  programme. 

(4) 

49 

12.The  Commission  should  give  consideration,  in  conjunction  with  Member  States,  industry  and  the 
research  community,  to  the  introduction  on  a  selective  basis  of  a wider  range  of  instruments  for  the 
conduct of IMT RTD activities. 

13.For  projects  where  it  is  agreed  between  proposers  and  the  Commission  that  patents  are  arelevant 

measure of innovation, the gaining of patents should be included in the objectives. 

14.The  Panel  commends  the  two  Post-Evaluation  Reports  (EUR  17457  and  EUR  16948),  and  the  1995 

External Monitoring Report for  consideration. 

4. 

50 

II. 

COMMISSION'S RESPONSE 

The Commision  is grateful  to the Five Year Assessment  Panel  for their thorough  and detailed  work 
in examining  activities under the IMT programmes during  1991-1995. The Report  is welcomed  and 
considered  fair  and balanced. The Panel's positive comments concerning the current programme  are 
acknowledged.  Continuous  improvements  are being  enacted  concerning  efficiency  of  management 
through  modern  informatics  and  streamlining  of  project  reporting,  financial  control  and  external 
technical  assistance.  These  actions  will  meet  the  concerns  expressed  over  limited  resources  both 
within the CRAFT activity and for the management of conventional RTD consortium  projects. 

The efficient  management  of the programme requires not only a full  transparency  of procedures and 
commitment  of  public  money,  but  also  the  achievement  of  realistic  timescales.  The  Panel  have 
rightly  identified  these as important.  Proposal  evaluation procedures  are under  constant  review.  To 
achieve  gains  in  time  will  essentially  require  accelerating  the  consultative  and  administrative 
procedures  following  proposal  evaluation  and  recommendation,  both  within  the  Commission  and 
with Member  States through the Regulatory  Committee. The Panel has identified  a recognised  need 
to increase SME participation.  This is being continuously  encouraged  and any further  move towards 
a more effective  decentralized  network  for  stimulating  SMEs will ensure  a high  level  of  "European 
dimension". 

As projects  under the IMT programme and its predecessors mature, exploitation  measures become a 
key part  of the effectiveness  of the programme. The increased  attention through  the  Accompanying 
Measures (and under Activity  3 of  Framework IV) is ensuring a significant  increase in the visibility 
of exploitation. 

In this context, the "Action Plan for Innovation  in Europe" (COM (96) 589) has important  messages 
for  the  take-up  of  results.  Transfers  of  intellectual  property,  know-how  and  experience  between 
different  industrial  sectors are essential. Patents and licensing,  additional  financial  intstruments, the 
effective  promotion  of  technology  transfer  deals  and  the  encouragement  of  SME  opportunities 
within the broader European context will all become crucial as the millenium  is approached. 

The  strengths  of programmes  such  as IMT  lie  in the  achievement  of  appropriate  balances  between 
basic  and  applied  research  on  the  one  hand,  and  on  the  other  between  a  generic  technical  content 
together  with  selected  vertical  concentrations  on  strategic  themes.  The  constant  accumulation  of 
experience  through  project  selection  and  management,  coupled  with  feedback  from  project 
indicators  and reviews  such  as the one just  completed  by this  Five  Year  Assessment  Panel,  should 
ensure a continued valuable contribution to Europe's industrial RTD base in the  future. 

51 

III. 

EVALUATION  PANEL 

Professor  Colin Seabrook 
After  an  extensive  career  in  engineering  and  the  recipient  of  a  number  of  awards,  Professor  Seabrook  is 
currently  visiting  professor  in  Engineering  Design  at  the  University  of  Bath.  He  continues  to  serve  on  a 
number  of  U.K.  and  Institution  of  Mechanical  Engineering  Committees  and  has  provided  important 
contributions  of  Advisory  Boards  within  the  Department  of  Trade  and  Industry  and  elsewhere  within  the 
U.K.  He  was  formerly  Group  Research  &  Development  Director  -  J.  H.  Fermer  &  Co.  Ltd.  Presently  a 
director  of CSMA  Ltd.,  he also runs an engineering  consultancy. 

Mr.  Frits Klostermann 
Trained  in  physics  and  mechanical  engineering,  Mr.  Klostermann  spent  most  of  the  last  32  years  of  his 
professional  career  at Philips Research  Laboratories,  Eindhoven,  eventually  responsible  before  his  retirement 
for  the  policy  of  Philips  International  Research  towards  the  emerging  European  cooperative  R&D 
programmes.  He currently  runs a small consultancy  company  specialising  in electronic device physics topics, 
management  studies and research  evaluation. 

Professor  Luis Oro 
Currently  professor  in chemistry  at the  University  of Zaragoza,  Professor  Oro has strong  links with  a broad 
range  of  R&D  topics  through  his  recent  position  as  Secretary  General  of  the  national  plan  for  Scientific 
Research  and  Technological  Development  in Spain.  He  has  wide  experience  of  OECD  Scientific  Policy  and 
of SME programmes  and requirements.  Professor  Oro is Vice-President  of the European  Science  Foundation. 
His  research  interests  include  organometallic  chemistry  and  homogenous  catalysis.  He  has  published 
extensively  in the  field. 

Dr. Jacques Lukasik 
As  scientific  director  of  the  Lafarge  Group  (a  major  construction  company  operating  in  45  countries),  Dr. 
Lukasik  has  considerable  experience  in  the  assessment  of  research  projects  and  interactions  with  the  wider 
scientific  and  technical  community  outside  his  own  company.  His  previous  experience  includes  Research 
Directorships  within  CNRS,  in particular  with  International  and  Industrial  relations.  He  has  held  a  series  of 
visiting  Professorships  in the USA,  Brazil  and  Egypt.  His technical  background  in research  was  originally  in 
fundamental  atomic and molecular physics, quantum optics, optoelectronics and lasers. 

Dr.  Hans Joergen  Pedersen 
Currently  Vice-President  for  Corporate  Technology  and  Research  at  Danfoss  in  Nordborg,  Denmark,  Dr. 
Pedersen  was  a  member  of  the  Industrial  and  Materials  Technologies  programme  review  panel  for  Brite-
Euram  II  (1991-1994).  His  presence  on  the  IMT  Five  Year  Assessment  Panel  provided  continuity  with  this 
earlier  review.  Dr.  Pedersen  has  had  extensive  research  management  experience  in  his  role  as  chairman  of 
the National Committee  for  the  Danish Materials  Science Programme,  and as a national  representative  on the 
European Organisation  for  Testing  and Certification  (EOTC).  Dr.  Pedersen's particular  research  interests  are 
in surface  coating technology,  microsystems  and Total Quality  Management. 

Dr. Ing. Heinrich  Bergmann 
Formerly  Director  of  the  Institute  for  Structural  Mechanics  at  the  DLR  (German  Aerospace  Research 
Establishment),  Dr.  Bergmann has combined his lifetime's  experience  with aerospace  materials  and  structures 
and  general  aeronautical  engineering  with  extensive  committee  advice  for  National  and  International  R&D 
programmes.  He  has  acted  as  a  consultant  to  the  European  Space  Agency  (ESA)  on  the  HERMES  Safety 
Advisory  Committee.  Dr.  Bergmann  is  Professor  of  Mechanics  at  the  University  of  Hannover,  visiting 
Professor  of  Aeronautics  at  California  Institute  of  Technology  and  visiting  Professor  of  Engineering 
Mechanics at Virginia  Polytech,  USA. 

52 

Mr. Frank Armstrong  (Rapporteur) 
Mr.  Armstrong,  formerly  Director  Aerospace  Vehicles  RAE  Farnborough  and  now  a  consultant,  has  held 
senior  posts  in  the  U.K.  Government  Aeronautical  Research  Establishments  which  involved  formulating, 
managing  and  coordinating  National  Research  and  Demonstrator  programmes  in  air  vehicle  technology.  He 
was  a member  of  the  review  panel  for  the  1990-1994  IMT programme  Area  3  "Aeronautics"  and  served  as 
rapporteur  for  the IMT Five Year Assessment Panel. 

4. 

53 

1996 FIVE-YEAR ASSESSMENT 

STANDARDS» MEASUREMENTS AND TESTING 

55 

L 

EXECUTIVE  SUMMARY 

The  specific  programmes  for  standards,  measurement  and  testing  in  the  Second,  Third  and  Fourth  EU 
Framework  Programmes  (FWPs)  have  undergone  major  changes.  The  BCR  programme  in  the  second 
FWP  was  a  closed,  basically  subscription-type  programme  that  became  the  Measurement  and  Testing 
(M&T)  programme  with  open  calls  on  specified  themes  in  the  third  FWP.  Finally  the  Standards, 
Measurement  and  Testing  (SMT)  programme  in  the  fourth  FWP  is  a  truly  open  call  programme  with 
broad  themes  and  a  small  part  covering  dedicated  calls  for  urgent  needs  of  the  European  standardisation 
bodies and  the  Commission. 

The  common  objectives  of  these  three  special  programmes  are  to  strengthen  the  functioning  of  the 
European  internal  market  and  the  competitiveness  of  the  European  industry  and  also  to  meet  the  needs  of 
society.  The  themes  of  the  programmes  have  been  enlarged  from  fabrication  of  Certified  Reference 
Materials  (CRMs)  and  measurement  support  to  Community  Policies  in  the  BCR,  to  measurements  for 
Quality  European  Products,  research  related  to  written  standards  and  technical  support  to  trade  as  well  as 
measurements  related  to  the  needs  of  society.  This  larger  scope  is  also  reflected  in  the  budgets  that  have 
increased  from  59.2  MECU  in  the  BCR  to  184  MECU  in  the  SMT  programme.  The  funds  for  the  Joint 
Research  Centre  (JRC)  are  not  included  in these figures  as the JRC  is not  included  in this  evaluation. 

The  European  added  value  is  very  high  in  the  BCR,  M&T  and  SMT  programmes  as  they  address 
areas  that  are  essential  for  the  EU  to  function.  The  importance  of  this  has  grown  continuously  and  is 
today  much  higher  than  at  the  time  of  the  BCR  programme.  Today  the  directives  and  standards  that  are 
developed  are  in  use  and  must  be  enforced.  The  necessary  conformance  assessment  depends  upon 
harmonised  and 
testing 
infrastructure.  In  addition  there  is  a  huge  backlog  in  the  development  of  standards.  All  of  this  needs 
strong support  from the specific  programmes  on standards,  measurement  and testing. 

traceable  measurements  and  a  uniform  and  reliable  measurement  and 

The  Panel  has  based  its  five  year  evaluation  of  the  BCR/M&T/SMT  programmes  upon  extensive 
interviews,  earlier  evaluations  of  parts  of  the  programmes  and  material  provided  by  the  programme  staff. 
The  resulting  recommendations  are  such that some of them can be  implemented  almost  immediately  by  the 
programme  management,  others  need  decisions  by  the  Commission  but  could  still  be  implemented  during 
the present  SMT programme  and  the  remaining  ones are  intended  as inputs to the fifth  FWP. 

The  role  of  the  programme  staff  has  evolved  considerably  during  the  assessment  period.  The  role  of 
scientific  officers  has  changed  from  scientific  involvement  to  a  more  administrative,  consulting  and 
monitoring  role.  This  has  increased  efficiency  and  more  projects  are  now  handled  per  scientific  officer 
than  before.  This  change  has  at  times  been  painful  and  the  management  has  not  always  been  able  to 
respond  to the changing  needs.  The project  teams  and  members  of the  Regulatory  Committee  see  the  SMT 
programme  presently  as  being  well  run  and  the  staff  as  helpful  and  responsive  to  needs.  The  Panel 
recommends  that  the  management  of  the  programme  is  developed  to  a  more  team-based  form  with 
much delegation  and involvement  of the whole  staff. 

Problems  are  seen  in  the  slow  and  complicated  general  procedures  of  the  FWP  in  the  Commission  for 
approving  project  proposals  and  letting  contracts.  The problem  is greatest  in  the  dedicated  calls  where  the 
time  from  proposing  a topic  to  the  signing  of  a resulting  project  takes  at  least  two  years.  It  is  felt  that  the 
procedures  that  are  created  to  ensure  impartiality  and  independence  may  create  a  false  sense  of  security. 
The  Panel  recommends  that  the  time  from  closing  a  call  to  signing  the  contracts  for  approved 
projects  shall  be  no  longer  than  six  months  and  a  special  approval  process  that  is  no  longer  than  three 
months  shall be  created  for  small and  short projects  for  solving urgent  European  needs. 

The  dissemination  and  exploitation  of  results  of  single  projects  was  not  a concern  in the  BCR  programme 
as  the  results,  like  developed  CRMs  and  intercomparisons,  had  a  clear  user  group  that  was  easily 
informed.  In  the  subsequent  programmes  this  question  needs  more  attention  as  the  results  of  the  projects 
can be used  widely  and  are from  more  diverse  areas.  Despite  only  36 of the  173 projects  in the  M&T 

57 

programme  having  been  completed,  the  output  of  projects  is  already  impressive.  The  Panel  recommends 
that  the  encouraging  start  to the  dissemination  and  exploitation  of  the  results  of  the  M&T  programme 
and in the  future  the  SMT programme  shall be continued  effectively  and more  systematically  and  be 
followed  up so that maximum  benefits  of the project  results are  obtained. 

The  concept  of  dedicated  calls  in  the  SMT  programme  is  seen  as  very  good.  The  Panel  believes  it  is  the 
right  concept  to  provide  solutions  to  specific  European  needs  in standardisation,  measurement  and  testing. 
The  Panel  recommends  that  the  dedicated  calls  concept  is  continued  and  further  developed  by 
allocating  more  funds  for  this  purpose,  increasing  the  number  of  European  groupings  that  propose  topics 
for  these calls  and  speeding  up the procedures  for  approving project  proposals  for  these calls. 

As  mentioned  earlier,  the  BCR/M&T/SMT  programmes  have  all  had  a  major  significance  for  the 
implementation  of  EU  policy.  The  panel  sees  a  strong  need,  that  has  been  expressed  from  many 
sources,  to  keep and strengthen  the  SMT in the  fifth  FWP as a separate  programme  or  in some  other 
way  organise  and  secure  that  it  remains  a  strong  unit  with  its  own  identity  and  funding.  The  overall 
objectives  of  the  present  SMT  programme  can  be  used  as  the  basis  for  the  next  programme  but  some 
revision  of  focus  and  objectives  as  well  as  additional  detailed  measurable  objectives  against  which  the 
progress  of the programme  can measured  are  recommended. 

The  panel  recommends  that  the  future  programme  for  standards,  measurement  and  testing  shall 
include  dedicated,  special  focused  and open calls.  With this combination,  all  types  of  European  needs  in 
this  field  can be  taken  care  of.  The  allocation  of  funds  to different  calls  and  themes  should  allow  for  some 
later  adjustment,  for  instance  by  the  Regulatory  Committee,  based  upon  the  experience  of  finished  calls 
and  emerging  new  needs.  Because  of the continuing  substantial  needs  in the fields  of the  programmes  it is 
recommended  that  the  total  financial  resources  for  the  future  programme  are  raised  to  300  MECU 
without  taking  into  account  the part  of the budget  allocated  to  the JRC.  The  relative  share  of  funds  for  the 
standards,  measurement  and testing programme  in the  FWP V should be clearly  higher  than  at  present. 

The  production  of  CRMs  has  been  a  valuable  contribution  to  European  metrology  in  the  programmes 
evaluated.  The  panel  sees that these  activities  also need support  in the future  programme.  This  should 
be organised  within  the  suggested  framework  in such  a way  that  the  important  European  needs  in the  field 
are  fulfilled  and the high  quality  of CRMs  is secured  and  maintained. 

The  Panel  had  to do  this  evaluation  with  a very  tight  schedule  of  only  three  months.  The  help  of  the  SMT 
programme  staff  was  very  valuable  and  without  it  the  work  could  not  have  been  completed  according  to 
the  given  schedule.  The  panel  members  have  all  actively  contributed  to the  report  and  the  conclusions  and 
recommendations  of the panel  are  unanimous. 

58 

II. 

COMMISSION'S  RESPONSE 

The  Commission  is very  grateful  to the  Five  Year  Assessment  Panel  for  its thorough  and  detailed  work  in 
examining  the  activities  of  the  Standards,  Measurements  and  Testing  (SMT)  programme  during  the  period 
1991-1995.  The  constructive  recommendations  made  by  the  panel  are  welcomed  and  will  be  taken  into 
account  during  the  4th  Framework  Programme  as  well  as  during  the  planning  of  activities  related  to 
standards,  measurements  and testing  in the 5th Framework  Programme. 

As pointed  out by  the panel,  the  SMT programme  has undergone  fundamental  changes  in  its scope  and  the 
way  that  it  operates  during  the  last  five  years.  These  changes  were  necessary  in  order  to  ensure  that  the 
programme  is managed  in the  same manner  as other  specific  industrial  related  research  programmes  in the 
4th  Framework  Programme.  This  evolution  is presently  continuing  towards  a  form  using  more  delegation 
and team  work. 

With  respect  to  other  recommendations  to  be  implemented  in  the  short  term,  the  Commission  fully 
recognizes  the  importance  of  ensuring  that  calls  for  proposals  are  not  over  subscribed,  of  reducing  delays 
in  the  project  evaluation  and  selection  process,  and  of  ensuring  the  exploitation  and  the  dissemination  of 
the  results  of  projects.  Appropriate  measures  are  being  taken,  and  will  continue  to  be  taken,  in  these 
directions.  One  important  development,  that  will  help  to  increase  the  exploitation  and  the  dissemination  of 
results,  is  that  the  progress  of  projects  is  now  being  monitoring  against  clear  quality  indicators.  The 
Commission's  preliminary  guidelines  for  the  5th  Framework  Programme  (Commission  documents 
COM(96)332  and  COM  (96)595),  echo  the  panel's,  recommendations  concerning  the  need  to  focus 
Community  research  efforts,  to  increase  the  flexibility  of  Community  research  programmes  in  order  to 
respond  to needs  as they  arise,  and to reduce the time taken  for  project  evaluation  and  selection. 

It  is particularly  pleasing  that  the  panel  has  recognized  the  merits  of  the  dedicated  call  system  in the  SMT 
programme.  The  system,  which  was  largely  experimental  when  it  was  first  set-up,  has  already  evolved 
considerably  and  will  continue  to  do  so  in  the  4th  Framework  Programme.  It  allows  Community  RTD 
activities  linked  to  standardization  to  be  precisely  and  efficiently  focused  where  the  real  urgent  needs  are, 
and  may  provide  a model  for  similar  systems to be set up in the  5th Framework  Programme. 

The Commission  fully  agrees  with the panel  on the paramount  importance  of standards,  measurements  and 
testing  for  the  future  competitiveness  of European  industry  in the global  economy  and  for  the general  well-
being  of  society. 

59 

III. 

EVALUATION  PANEL 

Professor  Freddy  ADAMS  (Rapporteur) 
He  is  currently  professor  of  chemistry  at  the  University  of  Antwerp  where  he  was  until  October  1995 
rector  and  chairman  of  the  University  Research  Council.  He  is member  of  a number  of  Belgium  Research 
Institutions.  His main  research  interest  focus  on environmental  chemistry  and material  science. 

Mr.  Michael  BROCK 
Trained  in  electrotechnic  and  acoustic  engineering,  Mr.  Brock  is  Director  of  the  Transducer  Products 
Division  at  Bruël  and  Kjaer.  In  addition,  he  acts  as  consultant  in  management  and  strategical  planning  of 
technological  complex  companies. 

Mr.  Pierre  CROON 
Mr.  Croon  is  currently  General  Director  of  the  Belgium  Institute  for  Standardization.  He  is  participating 
actively  in the  European  Committee  for  Standardization  (CEN-CENELEC). 

Mr.  Gôran LINDHOLM  (Chairman) 
Mr.  Lindholm  is  currently  Director  of  NORDTEST,  organization  for  testing  in the  Nordic  countries.  His 
previous  experience  includes  directorships  at  international  level  in  the  electronic  and 
information 
technologies  area. 

Dr.  Walter  RAULS 
Currently  managing  Director  of  the  Standards  Commission  for  testing  materials  in  DIN  (German  Institut 
for  Standardization).  He  is also participating  actively  in various CEN  working  groups. 

Mr.  Michel  ROUSSEAU 
Trained  in  mechanical  engineering,  Mr.  Rousseau  is  auditor  for  the  French  Committee  for  Accreditation 
(COFRAC). 

Dr. James  TWEED 
Dr.  Tweed  is  currently  technical  manager  for  programmes  on  testing  and  design  methods  at  AEA 
Technology  Harwell  (UK).  His technical  background  in research  is in polymers  and metal  composites. 

60 

1996 FIVE-YEAR ASSESSMENT 

ENVIRONMENT AND CLIMATE 

61 

I. 

EXECUTIVE  SUMMARY 

1.0 

Introduction 

The Five Year  Assessment  Exercise 
This  is the  Executive  Summary  of  the  1996  Five  Year  Assessment  of  Specific  Programmes  in  the  Fields 
of  Environment  and  Climate  Research  and  Technological  Development  (RTD).  It  covers  all  Framework 
research  activities  supported  by  the  Commission  in  these  areas  over  the  period  1991  to  mid-1996, 
including  activities  supported  by the Third  Framework  Programme  (1990-94),  and  all  activities  included  to 
date  in  the  Fourth  Framework  Programme  (1994-98).  In  compliance  with  COM  (96)  220  final  (22  May 
1996),  this Five  Year  Assessment  constitutes the final  evaluation  of the programme  launched  in the  field  of 
Environment  under  the Third  Framework  Programme. 

The  Assessment  Panel  charged  with  conducting  the  Five  Year  Assessment  met  on  several  occasions  to 
discuss  the  background  documentation  provided  by  the  Commission  services;  to  interview  Commission 
staff  and  representatives  of  organisations  with  an  interest  in  environmental  research;  and  to  draft  the 
Individual  Panel  Members  held  separate  discussions  with  senior  members  of  the 
Assessment  Report. 
scientific  community  across  Europe.  A  questionnaire  was  also  sent  by  Technopolis  Ltd.  to  all  project 
coordinators  of  the  Environment  Programme  (1990-94)  and  the  Environment  and  Climate  Programme 
In  total,  805  questionnaires  were  distributed  and  402  coordinators  responded  by  the  cut-off 
(1994-98). 
date  of mid-September  (50%  response  rate). 

Environment  and  Climate  Programmes 
The  Specific  Programme  in  the  Field  of  Environment  RTD  (1990-94)  was  aimed  at  contributing  to  the 
scientific  and  technical  basis  for  the  implementation  of EU policy.  It was divided  into the  following  areas 

Participation  in Global  Change  Programmes 
Technologies  and  Engineering  for  The  Environment 
Research  on Economic  and  Social  Aspects of Environmental  Issues 
Technical  and  Natural  Risks 

Some  554  projects  were  funded  over  the  course  of  the  programme,  which  had  a  project  budget  of  300 
MECU  and  an overall  programme  budget  of 315 MECU. 

The  Specific  Programme  in  the  Field  of  Environment  and  Climate  RTD  (1994-98)  has  aims  similar  to  its 
predecessor.  Amongst  other  things,  it is intended  to 

Contribute  to world  programmes  of  research  into global  change 
Satisfy  the  demands  of  subsidiarity  and  contribute  to  cohesion  via  the  promotion  of  scientific  and 
technological  cooperation  and  integration  between  European  universities,  research  institutes  and 
industry 
Act  in concert  with the  activities  of the Member  States 
Be of  a high quality  and strengthen the scientific  base in Europe 
Help  develop  the  scientific  knowledge  and  technical  competence  needed  to  fulfill  environmental 
policy  mandates 
Make  a  significant  contribution  to  growth,  competitiveness  and  employment  and  encourage  the 
participation  of  SMEs 

The programme  has a budget  of 567 MECU  and  is subdivided  into four  main work  areas 

Natural  Environment,  Environmental  Quality  and Global  Change  (47%  of the budget) 
Environmental  Technologies  (25%) 
Space  Techniques  Applied  to Environmental  Monitoring  and Research  (20.5%) 
Human  Dimensions  of Environmental  Change  (7.5%) 

63 

6. 

By  mid-1996,  313  project  awards  calling  for  an  EU  Contribution  of  246  MECU  had  been  granted. 
Another  22 Concerted  Actions  (EU  Contribution  of 5.3  MECU) had  also been  ratified. 

Higher  Education  Institutes  (HEIs)  and  Research  and  Technology  Organisations  (RTOs)  dominate  the 
programme.  For  Shared  Cost  Actions,  there  were  2,125  successful  institutions.  These  include  1,040 
HEIs;  863  RTOs;  123  firms;  and  99  other  organisations.  The  low  level  of  industrial  participation  is  not 
worthy. 

2.0 

The  Findings  of  the  Assessment  Panel 

The  Specific  Programmes  of  Research  and  Technological  Development  (RTD) 
in  the  fields  of 
Environment  (1990-94)  and  Environment  and  Climate  (1994-98)  have  made  important  contributions  to the 
development  of  environmental  science.  Programme  goals  were  highly  relevant  to  environmental  science 
agendas  both  in  Europe  and  globally,  and  the  work  conducted  had  the  potential  to  contribute  to  the 
evolution  of  environmental  policy.  Programme  execution  was  for  the  most  part  satisfactory,  and  goal 
attainment  in  terms  of  scientific  achievement  was  sound  in  most  areas  and  impressive  in  others.  There 
was  more  limited  impact  on  policy  development,  however,  and  low  industrial  participation  meant  that 
contributions  to European  competitiveness  were  marginal. 

Relevance 
The  need  for  a  programme  of  RTD  activities  in  the  field  of  Environment  and  Climate  remains.  An 
increasing  emphasis  on  sustainable  development  will  be  best  served  by  a  highly  visible  programme  of 
support  for  the  European  environmental  research  community  which  past  and  present  research  programmes 
have helped  to create. 

The  centre  piece  of  this  support  should  be  a  specific  framework  programme  which  combines  the  current 
Environment  and  Climate  and  MAST  programmes.  Efficient  mechanisms  are  needed  to  link  this  with  the 
environmental  research  needs  and  activities  of  the  Directorate  General  of  the  Commission  responsible  for 
the  Environment  (DGXI),  and  with  the  research  needs  of  other  relevant  Directorates  General.  It  should 
also  be  linked  with  the  work  of  other  bodies  and  agencies  such  as  the  European  Environment  Agency 
(EEA),  the  European  Space  Agency  (ESA)  and  the  European  Meteorological  Satellite  Organisation 
Improved  links  with  the  Joint  Research  Centre's  Environment  Institute  are  needed  to 
(EUMETSAT). 
make the  work  of this JRC  institute  more  visible to the European  environmental  science  community. 

The  goals  of  the programme  are  also  still  appropriate.  In particular,  there  is still  a need  to strengthen  and 
maintain  the  European  scientific  base  and  to conduct  policy-relevant  research.  Ensuring  the  participation 
of  industry  is  also  important  to  encourage  the  spread  of  environmental  best-practice  as  this  becomes  an 
increasingly  important  factor  in competitiveness. 

Programme  compositions  were  largely  in  line  with  programme  goals.  Environment  and  Climate  projects 
comprised  a mix  of  fundamental  and  applied  research  which  was  relevant  to  enhanced  understanding  and, 
to  a  more  limited  extent,  to  policy  development.  Participants  saw  projects  as  a  way  of  expanding 
existingknow-how,  creating  new  European  networks  and  developing  new  tools  and  techniques  via  access 
to  complementary  sources  of  expertise  in  other  European  organisations.  The  majority  of  projects, 
however,  were  not geared  towards  enhancing  industrial  competitiveness. 

Efficiency 
Work  within  the  environment  programmes  was  for  the  most  part  adequately  financed  and  conducted  in  an 
efficient  manner.  The programme  appears  to have been conducted  in a cost-effective  manner. 

Management  and  administration  of  the  programme  was  for  the  most  part  sound.  Some  problematic 
management  issues  should  be  resolved  with  the  appointment  of  a  new  Programme  Director;  general 
administration  was  satisfactory;  and  conduct  at  the  project  officer  level  was  commendable.  Participants 
were  appreciative  of the help given by project  officials  at all stages of the programme  life-cycle.  Most  felt 

64 

6. 

that  Commission  procedures  for  making  applications  were  clear  and  easy  to  follow,  though  many  still 
regarded  them  as too slow.  Late payment  was also a common complaint  across the research  community. 

There  were  serious  problems  in  terms  of  the  relationship  between  the  Commission  services  and  the 
Programme  Management  Committee  (COPEC).  The  transition  from  an  advisory  body  to  a  management 
committee  has  not  been  smooth  or  well  managed,  and  efforts  will  be  needed  in  the  future  to  establish 
productive  working  relations. 

A  number  of  other  administrative  problems  were  also  apparent. 
In  isolation  none  appeared  particularly 
severe,  but  a  timely  review  of  administrative  functions,  procedures  and  resources,  especially  staffing 
policies  and  practices,  would  help avert  problems  and maintain  current  performance  levels. 

The  range  of  support  modes  open  to  the  Commission  services  (Shared  Cost  Actions,  Concerted  Actions 
and  Accompanying  Measures)  was  satisfactory,  though  some  aspects  of  their  use  were  not.  Lack  of 
support  for  projects  involving  large  numbers  of  institutions  was  regretted.  Although  these  have  high 
coordination  and  communication  costs,  the benefits  of participation  often  outweigh  the  costs  and  a  flexible 
approach  to their  inclusion  is recommended.  There  was also  a marked  tendency  to  reduce  project  funding 
levels  with  no  corresponding  reduction  in  project  objectives  or  ambition.  This  should  be  avoided.  New 
transparent  procedures  for  assessing  project  proposals  were welcomed,  however. 

Effectiveness 
There  is little  doubt  that  participants  were  satisfied  with  project  achievements.  Immediate  goal  realisation 
occurred  in  terms  of  networking  and  financial  security,  whereas  knowledge  enhancement  was  seen  as  a 
medium-term  outcome. 

Project  coordinators  were  convinced  that  projects  had  realised  their  scientific  potential  and  that  the 
European  environmental  science  base  had  been  strengthened  as  a  consequence.  Networking  had  helped 
forge  a  dynamic  European  research  community  which  was  well  integrated  with  national  and  global 
research  agendas  and  associated  communities. 

Some  research  results had  fed  into policy  discussions,  though  overall  policy  impact  had  been  limited.  This 
is largely  a function  of  the  nature  of  the  work  conducted  and  the  complex,  lengthy  and  non-linear  ways  in 
which  scientific  evidence  feeds  into  and  affects  policy  environments  and  decisions.  Lack  of  policy  impact 
had  been  exacerbated,  however,  by  weak  links  between  the  DGXII  (Science,  Research  and  Development) 
research  community  and  policy-making  bodies  such  as  DGXI  (Environment)  and  other  Digs  concerned 
with environmental  issues. 

The  networking  aspects  of  the  programme  had  contributed  greatly  to  the  cohesion  aspects  of  EU  policy, 
though the  Less Favoured  Regions  still won a disproportionately  small  share  of  projects. 

level  of 

The  low 
to 
competitiveness  had  been  small.  In future  greater  efforts  will be needed  to  encourage  industrialists  to join 
projects  and  networks  and  to realise the  industrial  potential  of  projects. 

in  the  programme  meant  that  overall  contributions 

industrial  participation 

The  overlap  between  the  areas  of  interest  of  the  Specific  Programmes  and  national  programmes  is 
sufficient  to  ensure  that  the  Specific  Programmes  remain  relevant  to  national  needs,  but  not  so  extensive 
that they  can be considered  a substitute  for  national  efforts. 

All  the  available  evidence  points  to  genuine  additionality  and  subsidiarity,  with  participation  in  the  EU 
Specific  Programmes  adding  an  international  dimension  via  access  to  complementary  assets  not  attainable 
within national  boundaries.  Many  of the projects  concerned  with global  and  regional  issues could  not  have 
been undertaken  without  Commission  support. 

Dissemination  of  the  results  of project  outputs  to other  participants  in the programmes  and  to the  scientific 
community  at  large  had  taken  place.  Dissemination  to  policy-making  'end-users'  and  to  the  industrial 

(5) 

65 

community  had  been  more  disappointing.  New  mechanisms  to  ensure  this  occurs  need  to be  put  in place. 
This  could  involve  the  use  of  external  consultants  to  advise  on  dissemination  strategies  and  restructuring 
some future  activities  into User-focused  areas defined  in part by relevant user  communities. 

3.0 

Recommendations  of  the  Assessment  Panel 

Continuation 
The  Specific  Programme  of  RTD in  the  field  of  Environment  and  Climate should  continue  into  the  Fifth 
Framework  Programme.  Dispersing  environment-related  RTD  activities  into  other  programmes  is  not  a 
viable  or  desirable  option.  A  critical  mass  of  highly  visible  environment-related  research  is  needed  if 
sustainable  development  is  to  be  a  keynote  feature  of  the  Fifth  Framework  Programme.  This  does  not 
mean,  however,  that  all  environment-related  RTD  activities  should  be  centralised  in  DGXII(Science, 
Research  and  Development).  Other  Directorates  General,  especially  DGXI  (Environment),  DGVI 
(Agriculture),  DGVII  (Transport)  and  DGXVII  (Energy)  should  be  encouraged  to  continue  and  expand 
their  environment-related  RTD  activities,  with  improved  mechanisms  in  place  to  ensure  synergy  rather 
than  duplication. 

Integration 
The  Specific  Programme  of  RTD  in  Marine  Science  and  Technology  (MAST)  should  be  integrated  into 
an  expanded  Environment  and  Climate  RTD  programme.  The  arguments  in  favour  of  integration  out 
weigh  those  for  its  continued,  separate  existence.  In  particular,  there  is  a  need  to  strengthen  the  relative 
position  of  environment-related  RTD  in order  to ensure  that  sustainable  development  is central  to the  Fifth 
Framework  programme,  and  this  can  best  be  done  by  having  a critical  mass  of  flagship  RTD  in  a  highly 
visible  Specific  Programme. 

Networking 
The  emphasis  on  networking  within  the  Specific  Programme  of  RTD  in  the  field  of  Environment  and 
Climate  should  be  maintained  and  strengthened.  The  benefits  of  networking  across  Europe  were  highly 
prized,  readily  attained  and  much  appreciated  by  the  environmental  research  community.  These  benefits 
are  likely  to  be  short  lived,  however,  in  the  absence  of  sustained  efforts  to  nurture  a  culture  of 
cooperation,  collaboration  and  networking.  The  Panel  thus  recommends  a  continued  focus  on  network 
building  and  maintenance  between: 

Participants  within  individual  projects 
Participants  in different  projects 
All  interested  parties  in specific  topic areas,  especially  those  requiring  interdisciplinary  approaches 

Particular  efforts  should  be made to include  industry  in Environment  and  Climate  networks  and  projects. 

Segmentation 
The  programme  structure  should  segment  and  differentiate  between  User-focused  and  other  types  of 
research. 
Introducing  policy-driven,  customer-oriented  User-focused  areas  would  focus  programme 
activities  and  increase  the  likelihood  of  policy  relevant  work  having  appropriate  policy  impacts.  User-
focused  areas  should  be  limited  in  number;  account  for  no  more  than  20%  of  the  overall  budget;  have 
identifiable  user  communities  in  environmental  and  industrial  policy-making  circles;  and  a  high  policy 
relevance  in  the  short-  to  medium-term.  All  other  proposals  should  be  oriented  towards  the  medium-  to 
longer  term.  Possible  User-focused  areas might  include  activities geared  towards 

Global  issues and  concerns,  such as uncertainties  related  to the climate  change  agenda 
The  needs  of  multi-nation  regional  systems  and  regional  policy  makers,  such  as  regional  seas 
issues 
Some  of  the  specific  needs.of  industry  and  the  industrial  policy  community,  such  as  research  on 
cleaner  production  technologies 

66 

6. 

Some  of  the  specific  needs  of  DGXI(Environment)  related  to  the  implementation  of  the  United 
Nations  Framework  Convention  on  Climate  Change;  waste  treatment  and  disposal  technologies; 
and  public  perception  of environmental  issues 

Selection 
Selection  procedures  should  reflect  the  difference  between  User-focused  and  other  research  projects, 
with additional  criteria for  User-focused projects. 
In  addition  to  satisfying  excellence  criteria,  proposals 
in  User-focused  areas  should  specify  policy  relevance,  potential  users  and  ways  in  which  project  results 
might  be  exploited  in  policy  development,  and  this  information  should  be  taken  into  consideration  in 
selection  decisions.  Project  proposals  not targeted  at  User-focused  areas  should  not  be expected  to  contain 
such  detailed  information  on  potential  exploitation  routes,  though  its  presence  would  be  welcomed  given 
that  all projects  supported  within  the  Specific  Programme  should  be of  some  relevance  - direct  or  indirect 
- to environmental  policy  making. 

Shaping 
Individual  customer  communities  for  specific  User-focused  areas  (e.g.  DGXI  (Environment),  other 
DGs,  national  policymakers,  industry  etc.)  should  be  consulted  and  involved  more  extensively  in  the 
formulation  and  specification  of  research  agendas  and  work programmes.  This  would  focus  agendas 
more  closely  on  customer  needs  and  would  sensitise  these  user  communities  to  the  existence  and  potential 
utility  of  research  outputs.  For  the  remainder  of  the  programme,  the  Commission  Services  should  define 
its  scope  by  specifying  broad  areas  of  interest  and  concern  to  multiple  user  communities  -  including  the 
scientific  community  itself. 

Linkage 
Linkages  between  DGXII  (Science,  Research  and  Development)  and  DGXI  (Environment)  need  to  be 
improved,  particularly  in  the  policy-relevant  User-focused  areas.  Currently  DGXI  (Environment)  is 
unable  to  'see'  the  results  of  work  supported  by  DGXII  (Science,  Research  and  Development)  or  to 
appreciate  their  policy  relevance. 
In  part  this  can  be  rectified  by  better  'packaging'  of  results  and 
improved  communication  channels  and  mechanisms  between  the  two  DGs.  Critically,  however,  DGXI  has 
to  show  more  concern  and  play  a  greater  part  in  the  determination  of  the  research  agenda  and  work 
programme  for  a  specific  User-focused  area  geared  towards  its  short-  and  medium-term  needs  as  a 
customer  of  policy-relevant  research  outputs.  In  particular,  DGXI  should  make  public  its  medium-term 
work  agenda  and  its  implications  for  research.  Similarly,  other  DGs  with  an  interest  in  environmental 
issues  (e.g.  Agriculture,  Energy  and  Transport)  should  be  encouraged  to play  a  greater  role  in  the  setting 
of research  agendas  within  DGXII. 

Dissemination 
Dissemination  practices  should  be  improved  via  better  packaging  and  targeting  and  the  customised 
delivery  of  outputs  to  user  communities.  Specifying  User-focused  areas  and  involving  relevant  user 
communities  in  agenda  setting  will  undoubtedly  improve  the  prospects  for  the  take-up  of  policy-relevant 
research  results.  There  is  still  scope,  however,  for  delivery  mechanisms  to  be  improved.  User 
communities  are  interested  in  problem-solving,  and  thus  in  information  which  can  help  them  solve 
problems  -  whatever  its  origin.  Policy-relevant  results  of  environmental  research  supported  by  DGXII 
(Science,  Research  and  Development)  therefore  have  to  be  packaged  together  with  other  data  relevant  to_ 
specific  problems.  This  then has to be delivered  to well  specified  target  audiences  via  a number  of  routes, 
e.g.  publications,  brochures,  seminars,  workshops  etc.  DGXII  should  continue  to  explore  ways  of  doing 
this, possibly  using external  consultants to define  an effective  dissemination  strategy. 

Industry 
An  attempt  should  be  made  to  increase  industrial participation  via  a  sub-programme  oriented  towards 
The  evolution  and  implementation  of  technologies  with  reduced  environmental 
the  needs  of  firms. 
impacts  is  crucial  to  the  concept  of  sustainable  development,  yet  industrial  participation  is  currently  low. 
Ways  of  reducing  barriers  to  entry  and  making  participation  more  attractive  should  be  sought.  A  User-
focused  area  oriented  towards  some  of  the  needs  of  industry  is advocated.  The  alternative  sign-posting  of 

67 

industrial  applicants  to  other  Specific  Programmes,  e.g.  Industrial  and  Materials  Technologies,  is  not 
recommended. 

COPEC 
Attention  needs  to  be  paid  to  the  relationship  between  the  Commission  services  and  the  Programme 
Committee for  the  Specific  Programme  of RTD in the field  of  Environment  and  Climate(COPEC).  The 
change  in status  from  an advisory  body  to a management  committee  has  led  to a more  formal  and  strained 
working  relationship  with  the  Commission  services.  Action  is  necessary  to  restore  levels  of  mutual  trust 
and  cooperation.  The  emphasis  should  be  on  striking  a  balance  between  the  relative  powers  of  the 
Commission  services  and the  representatives  of national  governments. 

Financing 
Pressures  to  cut  individual  project  funding  and  spread  resources  too  thinly  over  a  larger  number  of 
projects  should be strongly resisted.  Project  participants  generally  receive  less than  the  amounts  requested 
in  their  proposals,  with  little  corresponding  reduction  in  the  scope  or  scale  of  project  objectives.  For 
many  this  involves  seeking  supplementary  funding  and  an effective  reduction  of  the  proportion  of  the  total 
cost  covered  by  the  Commission.  The  need  to  supplement  funds  suggests  that  original  requests  are  not 
overly  inflated,  and  that  the  scientific  integrity  of the  work  could  suffer  if these  requests  are  not  met  fully. 
To  avoid  this  happening,  the  Commission  services  should  scrutinise  suggested  cuts  carefully  and  resist 
pressures  to  spread  resources  too  thinly.  Access  to  advice  from  personnel  capable  of  making  realistic 
estimates  of  project  resource  requirements,  e.g.  those  with  experience  of  R&D  project  management, 
should  be be considered  via the appointment  of external  'reference  groups'. 

Administrative  Procedures  and  Resources 
A  review  of  administrative  procedures  and  resources  should  be  undertaken  across  the  Commission 
services dealing  with Environment and  Climate RTD activities. 
Complex  bureaucratic  procedures  lie  at  the  heart  of  complaints  about  slow  application  processes  and  late 
payments.  The  Assessment  Panel  was  also  told  that  expanded  workloads  were  not  commensurate  with 
modest  staff  increases  in  the  Commission  services  over  the  past  five  years,  and  that  overload  problems 
were  acute  in  some  programme  areas.  If  this  is  indeed  the  case,  immediate  step  child  be  taken  to  rectify 
the  situation.  Programme  management  has  to date  been  well  regarded  by  the  research  community,  and  it 
is important  not to let under  staffing  threaten  this  situation. 

68 

IL 

COMMISSION'S RESPONSE 

The  Commission  welcomes  this  independent  evaluation  of  EU  RTD  in  environment  and  climate  and  will 
take  the  recommendations  into  account  both  in  implementing  the  on-going  programme  under  the  Fourth 
Framework  Programme  and developing  the Fifth  Framework  Programme  for  the end of the  century. 

The  Commission  accepts  that  research  into  environmental  issues  at  the  EU  level  is  essential  to  respond  to 
societal  needs  and  aspirations,  as  well  as  to  address  fundamental  problems  arising  from  the  land,  sea  and 
air,  to  assure  a  safer  and  healthier  planet  within  the  context  of  sustainable  development  and  growth.  For 
this  reason,  a  major  theme  in  the  Fifth  Framework  Programme  should  be  unlocking  the  resources  of  the 
living  world  and  the  ecosystem  in  which  major  problems  such  as  our  future  water,  the  environment  and 
health,  better protection  from  catastrophes,  eco-management  and the urban  environment  will be  tackled. 

Networks  within  and  beyond  Europe  have  been  an  essential  element  to  successful  environmental  research 
and  European  teams  have  been  at  the  cutting  edge  of  addressing  our  global  challenges  through  their 
contributions  to  international  initiatives  such  as  the  Intergovernmental  Panel  on  Climate  Change  (IPCC), 
the  International  Geosphere-Biosphere  Programme  (IGBP),  the  World  Climate  Research  Programme 
(WCRP)  and  the  International  Human  Dimensions  of  Global  Environmental  Change  Programme  (HDP). 
These  links  will  continue  to  be  vital  in  the  search  for  solutions  to  issues  and  problems  not  found  or 
concentrated  solely  in Europe. 

The  need  to  involve  the  widest  possible  interest  groups  in the  shape  and  direction  of  future  environmental 
research  is a  key  element  to  securing  transparency  and  meeting  the  citizens'  needs.  This  will  continue  to 
be  a priority  for  the Commission  services  in the development  and  implementation  of  its  research  activities. 
Improvements  have  already  been  made  to  strengthen  links  between  the  operational  research  Directorates 
General  and  the  policy  making  Directorates  General  to  ensure  greater  synergy  and  coherency  and  this 
process  will be further  developed. 

Commission  services  recognise  that  continued  efforts  need  to  be  made  to  promote  the  opportunities  for 
industrial  participation  -  both  through  large  and  small  to  medium-sized  enterprises  -  in  the  research 
activities.  These  opportunities  are  being  taken  up  in  specific  areas  of  existing  research  activities  such  as 
environmental  technologies  and  space,  but  more  work  remains  to  be  undertaken  to  ensure  better  public  -
private  sector,  and  academic-industrial,  synergies  to  maximise  the  potential  opportunities  and  benefits 
from  collaborative  environmental  research.  Such  research  must  never  be  seen  as  exclusively  for  public 
sector  organisations  or  be  carried  out  only  through  public  financing.  Nevertheless,  the  Commission 
considers  that  the  establishment  of  an  industrial  subprogramme  specifically  directed  towards  the  needs  of 
industry,  as  proposed  by  the  Panel,  is  not  the  best  way  forward  to  assure  such  synergy  or  industrial 
involvement. 

Numerous  steps  are  already  taken  to put  EU  funded  environmental  research  to the best use.  Nevertheless, 
further  work  will  be undertaken  by  the  Commission  services  to  ensure  greater  awareness  and  visibility  for 
the  output  of  EU  environmental  research.  Solutions  need  to  be  found  to  ensure  that  the  vast  array  of 
results  come  in a digestible  form  as quickly  as possible to the attention  of those  who  can best use  and  need 
them. 

Some  brief  findings  are  made  by  the  panel  concerning  work  undertaken  by  the  Environment  Institute  of 
the  Joint  Research  Centre.  Whilst  the  panel  correctly  indicates  that  an  assessment  of  the  Environment 
Institute  was  not within the  remit  of the panel  and  has been undertaken  by  a separate  Visiting  Group  under 
the  chairmanship  of  Professor  Carlos  A.  Borrego,  the  main  report  nevertheless  refers  to  expenditure  by 
the  Environment  Institute  on  environment-related  work.  These  figures  are  misleading:  only  37%  of  the 
core  funding  of  the  Institute  (which  in  1995  totalled  43.5  MECU  and  not  46  MECU,  as  set  out  in  the 
report)  is  devoted  to  environmental  research,  the  remaining  63%  is  allocated  for  scientific  and  technical 
support  to  EU  policies  developed  and  implemented  by  various  Commission  services  (i.e.  primarily  the 
preparation  and  implementation  of  EU  directives).  The  amount  expended  by  the  Institute  on  research 

69 

6. 

during  the period  of  the  Fourth  Framework: Programme  will,  therefore,  represent  60-64  MECU  and not 
half  the  budget  of  the  whole  environment  and  climate  programme  (566.5  MECU),  as  indicated  by  the 
panel. 

Finally,  the  Commission  recognises  in  its documents  on preliminary  guidelines  for  the  Fifth  Framework 
Programme  that  changes  need  to  be  made  in  some  of  its  management  techniques  and  arrangements  to 
ensure more efficient  management. 

Issues  raised  in  the  evaluation  report  which  concern  the  management  of  the  research  activities  will  be 
taken forward  in this context.  The Commission  services do, however,  attach considerable  importance to 
the  need  for  constructive  and  open  dialogue  with  the  programme  committee,  whilst  preserving  the 
institutional  responsibilities  for  the  various  actors  in  the  research  process,  and  steps  are  already  being 
initiated to ensure the mutual trust and cooperation essential for the wellbeing of environmental  research. 

70 

IIL 

EVALUATION PANEL 

Dr.  Lea Kauppi  (Chairman) 
Director  General  of the  Finnish  Environment  Institute since  1995 

Prof.  Niels  Busch 
Director  of the  Danish  Russian  Energy  Research  Institute,  Moscow 
Former  Director  of RISO National  Laboratory  (Denmark) 

Prof.  Eloy  Garcia  Calvo (withdrew  due to ill health) 
Professor  of meteorology  of the Universidad  de Alcala,  Spain 
Manager  of the  Spanish  national  R&D programme  on climate 

Prof. Mike  Chadwick 
Director  (Europe)  Leadership pour  l'environnement  et le développement  (LEAD),  Switzerland 
Former  Director  of  the  Stockholm  Environment  Institute  and  Chairman  of  the  1995  Annual  Monitoring 
Panel  for  the  Environment  and  Climate  RTD  Programme 

Prof.  Wim Harder 
Scientific  Director  at  TNO  Institute  of  Environmental  Sciences,  Energy  Research  and  Process  Innovation, 
The  Netherlands 

Prof.  André  Lebeau 
Chair  of the  space and  space technology  programmes,  Conservatoire  national  des arts et métiers,  France 
Former  President  of CNES  and  Director  of Méteo  France 
Former  Head  of  ESA and  Scientific  Director  of  CNES 

Prof.  Francesco  Mauro 
Director  of  the  Environmental  Department  of  the  National  Agency  of  New  Technologies,  Energy  and 
Environment  (ENEA),  Italy 
Member  of Italian  Delegation  for  the conventions  on biological  diversity,  and desertification  and  drought. 

Mr.  Ken Guy  (Rapporteur) 
Director  of Technopolis  Limited,  an innovation policy  consultancy,  UK 
Former  senior  fellow  at the  Science  Policy  Research  Unit (SPRU) at Sussex  University. 

71 

1996 FIVE-YEAR ASSESSMENT 

MARINE SCIENCE AND TECHNOLOGY 

73 

I. 

EXECUTIVE  SUMMARY 

BACKGROUND 

Article  4.2  of  the  Decision  94/804/EC  on  the  specific  RTD  programme  in  the  field  of  marine  science  and 
technology  stipulates  that  the  Commission  shall  have  an  external  assessment  conducted  by  independent 
experts  of  the  activities  carried  out  within  the  fields/areas/domains  covered  by  this programme,  and  of  their 
management  during the five years preceding this assessment. 

'Five-year  Assessment'  covers  the  relevant  activities  carried  out  during  the  Third  Framework 
This 
Programme  (from  1991 to  1994)  and  those  carried  out  during  the first part  of  implementation  of  the  Fourth 
Framework  Programme  (until  early  1996).  Where  possible,  major  achievements  arising  from  the  Second 
Framework  Programme  (1987  to  1991)  are  also  reported.  This  assessment  subsumes  the  final  evaluation  of 
the MAST-II  specific  programme  under  the Third  Framework  Programme.  It  also  includes  an  assessment  of 
the early  stages of M AST-Ill, together with an analysis of the most significant  outputs from  MAST-I. 

The  Assessment  Panel  consisted  of  six  independent  experts  who  considered  the  Programme's  relevance, 
efficiency,  and  effectiveness 
in  delivering  its  objectives,  and  sought  to  highlight  examples  of  major 
achievements.  The  Panel  also  discussed  the  requirements  for  future  Community  research  in  marine  science 
and  technology. 

The  Panel  undertook  personal  interviews  with:-  selected  project  coordinators;  all  national  delegations  on  the 
MAST  Committee  (MAST-COM);  representatives  from  Commission  Directorates  DGIII,  DGXI  and 
DGXIV;  MAST  Programme  staff;  and  the  Secretary  of  the  EUROMAR  Programme.  The  Panel  also 
examined  the  results  of  a  standard  questionnaire  sent  to  all  project  coordinators  within  MAST,  as  well  as a 
wide range of literature  relating to the MAST Programme  and individual  projects. 

The  Assessment  Panel's  report  has  been  prepared  in  two  parts.  Part  1  (this  document)  is  the  Executive 
Summary  containing  the  main  findings  and  recommendations  arising  from  the  Panel's  deliberations.  Part  2 
(Final  Report)  describes  the  development  of  the  MAST  Programme;  discusses  the  impact  of  the  Programme 
on  a  wide  range  of  issues;  gives  examples  of  major  achievements;  presents  detailed  conclusions  and 
recommendations;  and includes various statistics related to the Programme. 

DEVELOPMENT  OF  THE  MAST  PROGRAMME 

The  MAST  Programme  has  developed  through  MAST-I  (1989  to  1991),  MAST-II  (1991  to  1994)  and 
M AST-Ill  (1994  to  1998).  MAST-I  was  essentially  conceived  as  a  pilot  programme  (budget  50MECU) 
covering  a wide  field,  with  a view  to developing  a successor  programme.  Under  MAST-II,  certain  activities 
begun  under  MAST-I  were  developed,  new  topics  were  introduced,  and  the  geographical  coverage  was 
expanded.  This was summed  up  in Annex  I to the Council  Decision of 7 June  1991 (91/351/EEC).  MAST-II 
had a final  budget  of  118MECU,  and covered  five  areas:- (I) Basic and Applied  Marine  Science;  (II)  Coastal 
Zone  Science  and  Engineering;  (III)  Marine  Technology;  (IV)  Supporting  Initiatives;  and  (V)  Targeted 
Projects.  An area of work  included  at a late stage was that of Risk  Evaluation. 

The  M AST-Ill  Programme  was  set  out  in  the  Council  Decision  of  23  November  1994  (94/804/EC),  and 
extends,  develops  and  refocuses  the activities carried  out under  the first two MAST  Programmes.  Its  general 
objective  is  "to  foster  the  scientific  knowledge  and  technological  development  necessary  to  understand  how 
marine  systems  function  at basin  scales,  in order  to prepare  for  sustainable  use  of the  oceans  consistent  with 
the  preservation  of  marine  environmental  quality  and  to  determine  their  role  in  global  change."  The  budget 
for  M AST-Ill  is  244MECU,  and  the  Programme  comprises  (A)  Marine  Science;  (B)  Strategic  Marine 
Research;  (C)  Marine  Technology  and  (D) Supporting  Initiatives.  In addition,  greater  efforts  are being  made 
to  involve  SMEs  (Small  and  Medium  Enterprises)  in  the  Programme,  and  stress  is  being  laid  on  parallel 
initiatives (eg. dissemination/ exploitation/  implementation). 

75 

CONCLUSIONS 

The  main  findings  which  the  Panel  obtained  from  their  assessment  are  given  below  under  the  general 
headings of the effectiveness,  efficiency  and relevance of the MAST Programme: 

Effectiveness  of MAST  Programme 

•  The  MAST  Programme  has  successfully  mobilized  a  European-wide  scientific  community  in  marine 
science,  through  the  development  of  networks  associating  a large  number  of  laboratories  from  different 
types of  organisations  and  different  countries.  The  Programme  has  stimulated  interdisciplinary  research 
projects  which  are  necessary  to  achieve  the  MAST  objectives,  and  it  has  increased  the  capacity  of 
Europe to participate  in large international océanographie programmes. 

• 

• 

Major  breakthroughs  have  been  achieved  on  a  wide  range  of  scientific  and  (to  a  lesser  extent) 
technological  issues.  The  Panel  is  convinced  that  most  of  these  major  results  could  not  have  been 
achieved  without  the  organisational  framework  and  interdisciplinary  approach  made  possible  by  the 
funding  opportunities  within the MAST Programme.  MAST  is the only one of the various  European  and 
International  programmes  which  has  both  the  mandate  and  the  budget  to  encourage  European 
collaboration  in major  research  projects. 

The  results  of  the  MAST  Programme  constitute  an  important  scientific  and  technological  asset,  which  in 
the  long  term  will  be  used  for  future  technological  developments,  and  for  management  and  policy 
initiatives.  The  strengthened  knowledge  base  developed  in  MAST  therefore  has  an  intrinsic  economic 
value  in the long term. 

The development  of the code  on data management  has greatly  enhanced  the  accessibility  and  value  of the 

data collected  during  all projects  in the MAST-II and M AST-Ill  Programmes. 

The  MAST  Programme  objectives  are  relevant  to  policies  in  many  sectors,  including  fisheries, 
environment,  transportation,  industry  etc.  It  is  difficult  as  yet  to  pinpoint  any  direct  benefits  of  the 
MAST  Programme  to  national  or  European  maritime  policies  -  indeed  most  Member  States  do  not 
possess  integrated  policies on marine  affairs.  However  MAST has strengthened  the basic knowledge  and 
the  scientific  expertise  required  for  their  development.  Within  the  Commission  there  seems  to  be  quite 
good  liaison  between  the  MAST  office  and  the  Fisheries  and  Environment  Directorates,  but  links  with 
the Industry  Directorate  and the  "Maritime systems of the future"  Task Force need to be  strengthened. 

The  MAST Programme  seems to have had  a significant  impact  on the coordination  between  national  and 
European  marine  science  R&D policies. In many  cases national marine  science  R&D policies have been 
based  largely  on  experience  gained  in  implementing  the  MAST  Programme.  There  seems  little  doubt 
also that the MAST Programme  has strengthened  cooperation amongst national groups. 

The  objective  of  enhancing  European  competitiveness  in marine  technology  has  only  been  achieved  to a 
limited  extent,  and  the  Panel  believes  that  this  is  due  to  at  least  two  factors:-  the  involvement  of 
industrial  companies  in  the  Programme  has  been  disappointing,  and  dissemination  of  the  results  of 
MAST to the industrial  community  has been  unsatisfactory. 

• 

The  Panel  welcomes  the  moves  in  MAST-III  towards  the  development  of  better  links  between  research 

workers and the users of their  results, whether they be industrial  companies or policy makers. 

Within  the  MAST  Programme  the  "Technology  stimulation  measures  for  SMEs"  have  had  a  rather  poor 
success  rate.  It  seems  that  either  the  availability  of  this  supporting  initiative  from  the  Commission  was 
not  disseminated  well  enough,  or  that  the  stimulation  measures  do  not  meet  the  needs  of  SMEs  with 
respect to their R&D activities. 

76 

Efficiency  of MAST  Programme 

•  The  Panel  was  disappointed  to  learn  that  very  few  of  the  members  of  MAST-COM  came  from  an 
industrial  background,  and  was  also  surprised  to discover  that  some  members  knew  little  of  the  MAST 
projects  in which their national  laboratories were involved,  or of the main results of the Programme. 

•  The  Panel  can  confirm  that  the  Commission's  Programme  Officers  in  the  MAST  office  are  highly 

regarded by the MAST  community. 

•  The  publication  by  the  MAST  office  earlier  this year  of  the  booklet  "Evaluation  of  Proposals"  has  been 
warmly  applauded.  However  the project  authorization  procedure  is still  not  clear  to  most  of  the  MAST 
applicants.  The  Panel  also  noted  that  contract  negotiation  almost  always  resulted  in a budget  reduction, 
usually  without a corresponding  reduction  in the project's  objectives. 

•  The  Commission  still  seems  to underestimate  the  magnitude  of  the  task  of  coordinating  MAST  projects, 
but  the  Panel  welcomes  the  moves  in the  MAST-III  Programme  to  allow  more  realistic  funding  of  the 
project  co-ordination costs. 

•  The  Panel  very  much  welcomes  the  greater  attention  which  is  being  given  to  dissemination  and 
exploitation  in MAST-III,  but believes that the supporting  initiatives  in this  respect  are  still  not  vigorous 
enough.  The  Panel  suggests  that  experts  or  organisations  with  a  broader  perspective  would  often  be 
more successful  than project  partners  in disseminating  the principal  outputs from  the MAST projects. 

Relevance of MAST Programme  objectives 

•  Most  of the  initial  objectives  of the  MAST Programme  (enhancing  European  competitiveness,  developing 
means  to  exploit  ocean  resources,  establishing  the  S&T  basis  for  management  and  protection  of  the 
marine environment)  are still valid because :-

• 

• 
• 
• 

Europe  is  facing  even  more  severe  competition  in  marine  industries  from  the  USA  and  Asiatic 
countries than it was ten years ago 
concerns about over-exploitation  of marine  living resources are more acute than five years ago 
possibilities of exploiting  deep-sea resources (oil and gas) are now  unfolding 
awareness  of the necessity  of a better  management  of European  coasts  is growing  and  is becoming  a 
critical  issue in most countries. 

RECOMMENDATIONS 

The principal  recommendations  of the Panel are: 

•  The  European  Union  must  maintain  a  Programme  with  a  strong  component  of  marine  science  and 
technology  if  Europe  wishes  to  exploit  the  oceans'  resources  while  protecting  its  marine  environment. 
The  Programme  should  include  all  themes  relevant  to  the  marine  environment,  such  as  coastal 
management;  fisheries  and  aquaculture;  offshore  energy;  exploitation  of  marine  resources;  maritime 
transport;  etc.  These  themes  have  a  considerable  overlap  in  their  requirements  for  science  and 
technology,  including:-  understanding  and  synthesising  natural  processes  and  systems;  observation  and 
monitoring  systems;  forecasting  systems;  exploration  and  exploitation  technology;  socio-economic 
modelling; and predicting  climate change. 

Extra  efforts  should  be made to involve  industry  (and  SMEs  in particular)  in the MAST  Programme.  The 
procedures  for  proposing,  evaluating  and  awarding  contracts  for  such  industrial  projects  need  re 
examining  with  the  purpose  of  achieving  considerable  simplification  and  acceleration,  and  to  make  the 
funding  arrangements more attractive. 

77 

•  Border  lines  between  MAST  and  EUROMAR  should  be  better  defined,  and  a  closer  collaboration 
between  the  two Programmes  should  be  achieved.  The panel  suggests  that  projects  approved  by  MAST 
(and  those  rejected  because  they  are  too  near-market)  should  be  required  to  submit  a  brief  project 
description to the EUROMAR  Secretariat for possible participation  in the EUROMAR  Programme. 

•  The Panel  recommends  also that borderlines  with other European  agencies  (such  as the European  Science 
Foundation  and  the  European  Space  Agency)  need  better  definition  so  as to  encourage  cooperation  and 
synergy,  and  that  closer  collaboration  should  be  established  between  MAST  and  other  international 
programmes. 

•  Further  specific  initiatives  should  be  taken  to  promote  dissemination  of  MAST  results,  such  as  (1) 
Production  of  a  regular  MAST  Newsletter,  (2)  Organisation  of  workshops  on  specific  themes  between 
scientists  and  engineers  and  technicians  from  industry,  (3)  Awards  of  contracts  to  technical  institutes  or 
engineering  companies  to  perform  a critical  review  of  the  relevance  and  potential  uses  of  the  scientific 
and technical  results of MAST projects,  and to disseminate them under an appropriate  format. 

•  The MAST  Programme  should  continue  to  include  a mix of  large,  medium  and  small  projects.  However, 
in  order  to  combine  the  advantages  of  large  and  small  projects,  the  Panel  recommends  that  the 
Commission  should  make better  use of the possibility  of  selecting  a suite  of  related  projects  (rather  than 
one  large  or  very  large  project),  whose  partners  would  be  encouraged  to  exchange  ideas  and  results  at 
workshops organised  by the Commission under the provisions of  'Accompanying  Measures'  funding. 

Some  recommendations  are  also  made  for  further  improvements  in  the  procedures  and  management  of  the 
Programme: 

•  For  larger  projects, 

the  scientific  coordinator  should  be  assisted  by  an  executive/administrative 

coordinator  appointed  and financially  supported  by the Programme. 

•  The  Commission  should  encourage  cooperation  between  European  partners  to  coordinate  the  use  and 
building  of  large  marine  research  facilities  (océanographie  vessels,  submarine  systems,  computers, 
hydrodynamic  tunnels etc.). 

•  The  process  of  selecting  projects  for  funding  should  be made  more transparent,  with  a full  description  of 

the criteria used,  and of their  relative weighting. 

•  The  Commission  should  examine  very  carefully  the  exact  purpose  of  the  contract  negotiation  stage,  and 
having  defined  its  purpose  should  make  this  perfectly  clear  to  the  project  partners.  The  contract 
negotiation  stage should  also be made as short as possible. 

•  Procedures  for 

'post-project 

review'  of  MAST  contracts  should  be  developed,  published  and 

implemented. 

•  The  Commission  should  examine  the  feasibility  of  introducing  methods  to  encourage  "new  blood" 

participation  in the MAST  Programme. 

Another  recommendation  refers  to  the  need  to  achieve  much  closer  cooperation  between  MAST  and  the 
Maritime Task Force. 

78 

II. 

COMMISSION'S  RESPONSE 

The  Commission  welcomes  the  independent  evaluation  of  the  Marine  Science  and  Technology  programme 
and  will  take  the  recommendations  into  account  when  implementing  in  1997-98  the  second  tranche  of  RTD 
projects  of the on-going programme. 

The  Commission  takes  careful  note  of  the  argument  that  a strong  component  of  research  on  marine  science 
and  technology  must  be  maintained  "if  Europe  wishes  to  exploit  the  ocean's  resources  while  protecting  its 
marine environment",.  Although the FP5  proposal may not include a dedicated programme  on the  European 
seas,  the various  themes highlighted  in the evaluation  report  are  likely  to be effectively  addressed,  especially 
through  2  of  the  priority  topics  put  forward  in the  Commission  document  "Inventing  tomorrow"  which  sets 
out  preliminary  guidelines  for  FP5  :  "Unlocking  the  resources  of  the  living  world  and  the  ecosystem"  and 
"Promoting  competitive and sustainable  growth". 

Attracting  more  participants  from  industry  has been  a long-standing  concern  for  Commission  services.  The 
second  general call for  MAST-III proposals  (deadline of  15 October  1996) reveals a clear shift  towards  topics 
in  technology,  and  consequently  an  increase  in  the  number  of  industrial  participants  is  expected.  More 
efforts  on  this  issue  will  be  made  in  the  future,  through  focusing  of  research  priorities,  more  systematic 
dissemination  of  results  of projects,  the  reinforcement  of  ad-hoc  schemes  in favour  of  SMEs,  and  continued 
participation  in the Task  Force  "Maritime  Systems of the Future".  It must be acknowledged  nevertheless  that 
the  very  concept  of  MAST  since  its  creation  in  1989,  with  an  important  component  on  studies  of  processes 
taking  place  in  the  sea,  precludes  strong  participation  from  industry  throughout  the  whole  range  of  the 
programme.  On  the  other  hand,  the  Commission  sees  great  scope  to  increase  the  socio-economic  relevance 
and  visibility  of  the  programme,  especially  in  sensitive  areas  such  as  coastal  zone  management,  or  the 
monitoring  and operational  forecasting  of coastal seas. 

A  special  feature  of  the  MAST  programme  is the  existence  of  regional-scale  projects,  in  the  range  of  8-12 
MECU  of  EU  contribution.  Through  these  activities,  the  Commission  has  experience  both  in  monitoring 
large  single  projects,  centrally  coordinated  by  a  prime  contractor,  and  in  clustering  normal  size  contracts 
around  common  objectives.  The  Commission  endorses  the  panel's  recommendation  to  resort  more  to  the 
latter scheme. 

Linkage  with  international  programmes  and organizations  is a vital  necessity.  MAST has  already  contributed 
to the objectives  of  i.a.  IGBP core  projects  and  intends to take this further.  The  MAST  office  is also  active 
in  EUROMAR  and  has  begun  a  fruitful  collaboration  with  EMaPS,  the  secretariat  of  the  European  Marine 
and Polar Boards newly set up under the auspices of ESF. 

By  implementing  a number  of  supporting  initiatives,  the Commission  has begun  to link up  European  marine 
scientists around  common concepts and practices  in data management  and  in modelling.  The  need to address 
on  a  significant  scale,  by  other  appropriate  initiatives,  the  coordinated  use  of  large  research  facilities,  is 
acknowledged. 

79 

I I I. 

EVALUATION  PANEL 

Prof. Henk POSTMA  (Chairman) 
Background: chemistry.  Former  positions: professor  in physical  and  chemical  oceanology  at  the  University 
of  Groningen  (1963-86);  director  (1965-88)  of the  Netherlands  Institute  for  Sea  Research  (NIOZ);  chairman 
for  10 years  of  the  Netherlands  Committee  for  Sea Research;  chairman  1973-75  of  the  Scientific  Committee 
on  Océanographie  Research  (SCOR).  Member 
and  former  chairman  of  the  Dutch  SCOPE-IGBP 
Commission;  member  of the Dutch Climate Commission.  Officer  in the Order  of  Orange-Nassau. 

Mr. Michael W.  OWEN  (rapporteur) 
Background: engineering  science.  Chartered  engineer.  Chairman  1991-95  of  the  Programme  Management 
Committee,  coastal  engineering  research  programme,  Engineering  and  Physical  Sciences  Research  Council. 
)  of  the  Coastal  Engineering  Advisory  Committee,  Institution  of  Civil  Engineers. 
Member  (1992- 
Chairman  of  the  steering  group  of  the  MAFF  research  project  on joint  probabilities  of  waves,  swell,  surges 
and tides. 

Dr. Josefina  CASTELLVI 
Background:  biological  sciences.  Research  professor  at  "Consejo  Superior  de  Investigaciones  Cientificas" 
(CSIC).  Main  former  positions: Vice Secretary  General  for  national  coordination of RTD programmes  in the 
"Comision Interministerial  de  Ciencia  y Tecnologia"  (1995);  director  of the  "Instituto  de  Ciencias  del  Mar", 
CSIC  (1994);  Spanish  representative  in the  Antarctic  Treaty  (1988);  manager  of  the  National  Programme  of 
research  in Antarctica  (1989-1994). 

Dr.  Kari  KVESETH 
Background:  environmental  chemistry.  Since  1993:  director  of  the  Division  of  Science  and  Technology, 
Research  Council  of Norway.  Chairperson  of the Council,  Norwegian  Space Centre.  Member  of the  Board, 
Norwegian  Academy  for  Technical  Sciences.  Member  of  the  Armed  Forces  Council  for  research  policy. 
Main  previous  honorary  positions:  chairperson  of  the  Board,  Institute  of  marine  research;  member  of  the 
National Committee for  environmental  research. 

Prof. Pierre PAPON 
Background: physics,  applied  mathematics.  Professor  of physics  at  Ecole  Supérieure  de  Physique  et  Chimie 
Industrielles  (ESPCI),  Paris.  Chairman  of the board  of Observatoire  des Sciences et Techniques  (OST) since 
1990.  Member  of  the  Executive  Council  (since  1992)  and  of  the  Bureau  (since  1995)  of  the  European 
Science Foundation.  1989-95: president-director  general of IFREMER.  1982-86: director  general  of CNRS. 
Former  board  member  of  Agence  Française  pour  la  Maîtrise  de  l'Energie,  Commissariat  à  l'Energie 

Atomique,  Compagnie  Générale  d'Electricité. 

Mr. Johannes POST 
Background:  oceanography,  and  also:  geophysics,  meteorology,  theoretical  physics,  applied  mathematics. 
Experience  in  off-shore  and  deep-sea  surveying,  computer  simulation  models  and  computer  software  for 
marine  applications.  Founding  member  and  managing  director  of  Hydromod  Service  GmbH,  Hannover, 
since  1989.  Founding  member  of  OSAT GmbH,  Heist  (1989).  Board  member  of the  GEOMAR  Enterprise 
Association. 

80 

1996 FTVE-YEAR ASSESSMENT 

BIOTECHNOLOGY 

81 

I. 

EXECUTIVE  SUMMARY 

Introduction 

The  period  of  this  evaluation  is  the  five  years  from  mid  1991  to  mid  1996.  The  framework  II  BRIDGE 
programme  covered  the  period  from  1990  to  1993  with  an  expenditure  of  lOOMECU.  Biotech-I  in 
framework  III extended  from  1992 to  1994 and had  a budget  of  186MECU.  The  framework  IV  Biotech-II 
programme  extends  from  1994 to  1998 with a budget  of 588MECU.  Thus the period  under  review  covers 
the  end  of  the  BRIDGE  Programme,  all  of  Biotech-I  and  the  first  part  of  Biotech-II.  The  BRIDGE 
Programme  has been  evaluated  separately  and a comprehensive  report  is available.  The BRIDGE  report  is 
used  as  the  first  frame  of  reference  and  changes  that  occurred  in  Biotech-I  are  commented  on.  It  is  too 
early  to reach definitive  conclusions  about  Biotech-II.  Thus this evaluation  has concentrated  mainly  on the 
activities  of  Biotech-I  but  with  some  comments  about  changes  that  are  noted  in Biotech-II.  It  incorporates 
the final  evaluation under  the Third  Framework  Programme. 

The  main  findings  of  the  evaluation 

1.  The  initial  objectives  of  the programme  remain  valid  in  spite  of  rapid  changes  that  have  taken  place  in 
the  science  and  technology.  Close  attention  to  the  relevance  of  the  programmes  needs  to  continue  since 
Molecular  Genetics  and  its  associated  disciplines  now  provide  basic  tools  in  almost  all  the  bio-industries. 
It  is  important  to  strike  a balance  in  the  objectives  between  the  long  term  commitment  required  to  realise 
the  benefits  and  the  need  for  flexibility  in  response  to  the  advance  of  the  basic  science  and  technology  or 
the demands  of special  situations. 

2.  Whilst  remaining  true  to  its  initial  objectives,  the programme  has demonstrated  the  flexibility  to  change 
and  has  therefore  evolved  over  time.  The  Panel  approves  of  this  ability  to  change  but  new  targets,  or 
changes  of  emphasis  within  existing  targets,  need  to  be  publicised  earlier  and  more  widely  among 
scientists  in  the  community  so  that  they  can  be  aware  of  how  their  own  field  of  expertise  matches  the 
evolving  priorities  of the programme  areas. 

3.  One change  that the Panel  noted  was that  Biotech-I  was oriented  more towards  basic  research  compared 
to  the  earlier  and  following  programmes.  Whether  due  to  this  or  some  other  cause  there  was  a  lower 
participation  by  industry  in that  programme. 

4.  The  selection  criteria  are  followed  in the  main  and  continue  to  contribute  to the  building  of  a  European 
scientific  community  although  it appears that  the cohesion  criterion  is not  so obviously  used.  There  is now 
little  or  no  evidence  of  resistance  to  trans-national  collaboration  among  scientists.  However  it  is  still  the 
case  that  many  projects  are  not  truly  interdependent  being  more  a  collection  of  largely  independent  sub-
projects.  While  such  projects  do  frequently  contribute  to  skill  and  technology  transfer  between  partners, 
the opportunity  to deepen  further  the relationships  between  European  laboratories  may  be being  missed. 

5.  With  the parameters  within  which  it operates,  the programme  is highly  cost  effective,  having  an  impact 
far  greater  than  its  expenditure  would  predict  when  compared  with  the  total  European  effort  in  this  field. 
To achieve  this  it relies on the  infrastructure  and basic science  support  of the  National  programmes. 

6.  The  performance  of  the  Scientific  Officers  continues  to  receive  favourable  comments  but  there  are  a 
number  of  areas  in  which  the  management  process  could  be  improved.  These  are  discussed  and 
recommendations  made.  Of  particular  importance,  however,  is  the  delay  in  starting  projects  after  the 
initial  funding  decisions  are  made  since  this  can  affect  the  competitiveness  of  industry  and  may  inhibit 
industrial  participation. 

7.  As the  programmes  have  enlarged,  the  role  of  the  Programme  Committee  has  changed.  There  is  less 
clarity  in  the  relationship  of  the  Committee  to  the  evaluating  panels.  Although  the  Committee  approves 
policy,  its  role  in  implementation  has  become  more  passive.  While  there  is  no  case  for  it  to  become 

83 

involved  in evaluation  there  is a need  to ensure  that the committee  can monitor  the link between  policy  and 
implementation. 

8.  The  short  term  performance  of  the  programme  assessed  by  its  publication  record  seems  to  be  good, 
even  though  all  publications  may  not  yet  have  appeared.  However,  real  time  analysis  of  the  progress  of 
each  project  against  its  own  objectives  would  be  a  valuable  addition  to  performance  evaluation  and  new 
project  selection  criteria.  Effects  on  industrial  competitiveness  and  results  in  the  market  will  not  be  seen 
until  some  time  after  the  end  of  a  programme.  It  will  be  necessary  to  evaluate  these  features  on  a  much 
longer  time  frame.  However,  it  is already  clear  that the  number  of patents  filed  by  projects  in Biotech-I  is 
very  low. 

9.  The  Panel  has  found  that  some  companies  have  difficulties  with  the  rules  of  ownership  of  the  results  of 
projects  that  may  have  an  impact  on  exploitation.  Since  this  may  not  be  a  universal  problem  an 
investigation  is  needed  to  assess  its  potential  impact  and  guide  the  development  of  procedures  that  will 
eliminate  it. 

10.  The  Panel  has  found  evidence  of  a need  for  a broader  consultation  with  industry  in the  formulation  of 
programme  objectives.  In particular,  the  high  level  advice  provided  by  IRDAC,  the  SAGB and  the  major 
trade  associations  may  not  adequately  represent  the  experience  of  small  companies  nor  of  the  scientific 
middle  management  of large  companies. 

11.  The  Panel  has noted  that  there  are differences  in the needs and methods  of operation  of small  and  large 
companies.  The  structure  of  the  present  programmes  seems  to  support  the  needs  of  large  companies  well 
since  they  can  take  a  longer  term  view.  Small  firms  need  mechanisms  that  support  their  nearer-market 
approach.  Separate  instruments  that  address  their  knowledge  requirements  more  directly  should  be 
investigated. 

12.  As  a result  of  its own  experience,  the  Panel  is very  concerned  about  the  inefficiency  of  the  procedure 
for  external  evaluation.  The  period  preceding  the  appointment  of  the  Panel  was  subject  to  very  extensive 
administrative  delays  that  left  the  Panel  a mere  three  months  in which  to perform  its task.  This  means  that 
the  analysis  could  not  be  based  on  independent  data  collected  at  the  instigation  of  the  Panel  and  at  its 
worst,  could  call  into question  value of the whole  exercise. 

13.  The  Panel  has  recognised  a number  of  major  achievements  of  the  programme  such  as  the  Lactic  Acid 
Bacteria  project,  the  Lipase  project  and  the  genome  sequencing  and  structure  projects,  particularly  the 
It  is  noticeable  that  some  projects  that  are  leading  to  industrial 
Yeast  Genome  sequencing  project. 
application  require  support  in  more  than  one  programme  to  achieve  success.  The  mechanism  of  the 
continuation  of  support  through  competitive  bids  of  a  project  that  has  already  been  approved  and  is  a 
success,  may  need  to  be  reviewed.  The  real  time  monitoring  of  projects  in  progress  could  be  of 
importance  in this  respect. 

14.  The  Panel  has noted  that  there  is very  little co-ordination  between  the  EU  and  National  programmes  in 
biotechnology,  and  even  less  between  the  various  programmes  of  the  Commission.  Some  duplication  or 
overlap  between  Community  and  National  programmes  is unavoidable.  It  even  has  some  advantages  and 
it  may  be  essential  if  Community  programmes  continue  to  build  on  the  infrastructure  and  basic  scientific 
capabilities  resulting  from  the  National  programmes.  On  the  other  hand,  the  lack  of  co-ordination 
between  the  biotechnology  components  of  the  various  EU  programmes  is  inefficient  and  wasteful  of 
resources.  The  best  way  to  overcome  this  is  to  combine  the  biotechnology  aspects  of  all  three  current 
programmes  (FAIR,  BIOMED  and Biotech-II). 

15.  The  EU  programmes  are  continuing  to  support  the  scientific  base  in  that  a  real  scientific  community 
has  emerged  and  continues  to be  strengthened.  The  programmes  also  add  value  to  the  European  research 
effort  through  projects  that  could  not be co-ordinated  effectively  by  National  programmes  alone. 

Principal  recommendations 

84 

The  Panel  has  made  a  set  of  recommendations  under  five  headings  that  are  set  out  in  chapter  IV  of  this 
document  .  The ten principal  recommendations  from  this set are given  below. 

1.  Since  there  is  evidence  that  the  distribution  of  biotechnology  topics  between  three  programmes 
(Biotech-II,  Agriculture  and  Fisheries  and  BIOMED-II)  is  resulting  in  a  loss  of  synergy  we  recommend 
that  the  biotechnology  aspects  of  the  current  programmes  be  combined  in  future  programmes  but  with 
clear  linkages  to the user  sectors  concerned. 

2.  As  a  means  of  addressing  the  special  problems  of  small  companies  and  to  stimulate  processes  that  can 
lead  to  the  initiation  of  more  start-up  companies,  we  recommend  the  development  of  financial  platforms, 
analogous  to  or  alongside  the  present  industrial  platforms,  with  the  aim  of  channelling  private  sector 
funding  into the exploitation  of opportunities  generated  by EU  programmes. 

3.  The  Panel  is  of  the  opinion  that  the  procedures  for  consulting  with  industry  are  too  narrow  and  at  too 
high  a  managerial  level.  We  recommend  that  the  consultation  process  should  be  modified  to  take  into 
account  the views  of middle  management  industrial  scientists,  the views  of users  and  of  the managers  from 
small  companies. 

4.  The  Panel  confirms  its agreement  with  the  conclusion  of previous  evaluations  that  training  is one  of  the 
most  important  features  of  the  programmes.  We  note  that  while  the  training  component  of  Biotech-I  was 
not  improved  compared  to  BRIDGE,  that  of  Biotech-II  up  to  the  present,  is  better.  We  recommend  that 
the training  activity  of Biotech-II  be carefully  monitored  to maintain  this  improved  performance. 

5.  The  Panel  considers  the  process  of  project  evaluation  and  selection  to  be  fair  and  supports  the  strong 
adherence  to  peer  review  and  the  ranking  of projects  by  scientific  quality.  The  Panel  makes  a  number  of 
recommendations  to  improve  the  process  based  on  thé  comments  of  scientists  that  have  been  directly 
involved.  Of these,  probably  the most  important  is that  greater  emphasis  should  be  given  to the  quality  of 
the  feedback  notes.  This  is  one  of  the  main  contacts  between  Commission  staff  and  the  scientific 
community  and  it can be of  immense value  in the preparation  of future  applications. 

6.  The  Panel  have  repeated  the  observation  of  earlier  panels  that  the  internal  Commission  procedures  for 
approving  project  selection  following  scientific  peer  review  evaluation  are  unacceptably  long  and  severely 
reduce  the  competitive  impact  of  the  programme.  We  recommend  that  a  radical  review  of  procedures 
should  be undertaken  to eliminate  delays  and to reduce  the administrative  burden  on Commission  staff. 

7.  The  Panel  has  observed  that projects  that  support  infrastructure  activities,  such  as databases  and  culture 
collections,  suffer  from  difficulties  due  to  the  rigid  contractual  arrangements  of  the  Commission  and  the 
need  for  the  continual  renewal  of  short  term  contracts  for  the  support  of  a  long  term  function.  We 
recommend  that  a  new  mechanism  be  devised  to  provide  long  term  contracts  for  infrastructure  services 
including  built  in reviewing  procedures  to check  their performance  as a justification  for  the  continuation  of 
funding. 

8.  The  Panel  has  received  evidence  that  the  Programme  Committee  has  become  remote  from  the  process^ 
of  project  selection  and  that  this  separation  may  permit  inadvertent  changes  of  policy  due  to  the  spread  of 
projects  selected.  We  recommend  that the  Programme  Committee  be  given  a quality  assurance  role  in the 
process  of  project  selection  so  that  the  link  between  policy  and  practice  is  fully  monitored  by  the 
Committee. 

9.  As  a  result  of  its  experience  in  the  initiation  of  this  evaluation  and  because  of  its  knowledge  of  the 
preceding  administrative  delays,  the Panel  is seriously  concerned  that  the  actual  evaluation  of  programmes 
is  not  given  proper  priority.  We  recommend  an  independent,  in  depth  review  of  the  external  evaluation 
process  compared  to  the  Commission  objectives,  to  include  the  difficulties  experienced  by  this  Panel  as a 

The full  report  of the panel  can  be obtained  separately. 

85 

result of the implementation of the current exercise and the use that could be made of real time monitoring 
of projects  in programme evaluation. 

10.  The present practice leaves evaluation panels, and perhaps others, unaware of the use that is made of 
the results of their  efforts.  We recommend  that the Commission  should  make an explicit public response 
to the recommendations made by external evaluating panels with reasons for their acceptance or rejection. 

86 

II. 

COMMISSION'S  RESPONSE 

The  Commission  wishes  to  express  its  appreciation  to  the  panel.for  its  report  on  the  assessment  of  the 
activities  of  the  Biotechnology  Programmes  for  the  last  five  years  (1991-1995).  The  Commission 
recognises  the  fact  that  the  report  was  carried  out  under  a  pressure  of  time,  however,  it  regards  the 
outcome  as  fair,  accurate  and  as  having  reached  balanced  conclusions  and  important  recommendations. 
The Commission  will comment  on the main  recommendations: 

-  As  the  panel  recommends,  the  biotechnology  aspects  of  current  programmes  should  be  combined  in 
future  programmes.  The  future  programme  covering  Life  Sciences  and  the  Ecosystem  can  focus  on  the 
opportunities  clarified  in  the  past,  so  as  to  reap  the  benefits  of  a  concentrated  effort  combining  all 
resources  and  means  which may be obtained  in a single unified  programme. 

-  In  addressing  the  special  problems  of  SMEs,  a set  of  'stimulation  measures  for  SMEs'  has  already  been 
introduced.  There  seems  to  be  a  general  appreciation  for  these  measures  and  the  way  in  which  they  are 
implemented  in  the  Biotechnology  programme.  A continuous  evaluation  and  further  development  of  these 
measures  is needed  in order  to optimise  them. 

industry, 

-  Concerning 
the  Commission  receives  advice  from  IRDAC  (Industrial  Research  and 
Development  Advisory  Committee),  especially  for  the  preparation  of  the  Framework  Programmes.  In 
addition  to  this  institutional  and  strategic  advice  provided  by  IRDAC,  the  Commission  services  receive 
specific  advice  from  other  industrial  groups  throughout  the  crucial  steps  of  programme  preparation  and 
implementation. 

-  As  far  as  training  is  concerned,  the  Commission  is  making  every  effort  to  maintain  the  current  success 
rates which  are the result  of the recommendations  made by  the evaluation  of  BRIDGE.  In addition,  we  are 
devising  methods  for  monitoring  the quality  and outcome of the increased  number  of grants. 

- The  Panel  considers  the project  evaluation  and  selection  to be  fair.  In addition  it gives  great  emphasis  on 
the  quality  of  the  summary  consensus  reports  which  serve  as a feedback  communication  mechanism  to  the 
applicants.  The  Commission  takes 
this  remark  very  seriously.  Currently,  the  quality  of  the  summary 
consensus  reports  have  significantly  improved  compared  to  the  reports  of  the  previous  call  for  proposals. 
The  importance  of  adequate  feed-back  to proposers  is  emphasised  to  the  external  evaluators  participating 
in the different  peer-review panels. 

- Since one of the main goals of the Framework  Programme  is to enhance the competitiveness  of  European 
Industry,  measures  are  being  taken  in  order  to  increase  the  transfer  of  successful  research  results  into 
marketable  products.  The  involvement  of  industrial  experts,  as  well  as  experts  from  the  industrial 
platforms,  in  the  preparation  of  the  programme  and  in  the  selection  of  projects,  constitute  means  for  the 
promotion  of  the  industrial use of  research  results. 

-  The  Panel  made  the  observation  that  the  internal  Commission  procedures  concerning  project  evaluation 
and  selection  are  rather  long.  The  real  delays  for  the  Commission's  decision  after  peer  reviews  are 
approximately  two  and  a  half  months.  The  Commission  is  making  a  great  effort  to  reduce  it  further,  but 
without  detriment  to the quality  and  fairness  of the  work. 

-  The  Panel  suggested  the  improvement  of  the  coordination  among  EU  and  national  programmes  in 
biotechnology.  The  Commission  has  already  taken  measures  to  do  this,  aiming  at  increasing  programme 
efficiency,  synergy  and  added  value.  An  example  is  the  pilot  study  dealing  with  protein  engineering  and 
structural  biology  activities  in Europe,  conducted  by the Biotechnology  programme. 

-  The  Panel  has  received  evidence  that  the  Programme  Committee  has become  remote  from  the process  of 
project  selection.  According  to  the  Commission,  there  is  no  reason  to  believe  that  the  role  of  the 
committee  has  changed.  Indeed,  Article  7  of  the  Council  decision  stipulates  what  the  committee's  role 

87 

consists of.  This has not significantly  changed over time, except for the threshold of financial  commitment 
under  which  the  committee's  opinion  is  not  required  (now  0.5  MECU,  0.3  MECU  before).  The 
Committees' opinion is now also sought on training grants and all accompanying measures. 

- The Panel  is seriously  concerned  that the actual  evaluation of Programmes  is not given proper  priority. 
This  statement  is based  on  the  fact  that  the  administrative  delays  are  by  far  longer  than  the  actual  time 
given  to  the  Panel  to  finalise  their  exercise.  The  Commission  acknowledges  and  regrets  the  time  lost 
before  the  panel  could  carry  out  its  work.  It  is  to  be  expected  however  that  further  assessment  and 
monitoring  exercises  will  benefit  from  the  fact  that  monitoring  and  assessment  have  now  become  a 
systematic part  of the daily  routine of the Commission's  tasks  in the framework  of the  implementation of 
RTD programmes. 

-  The  Panel  suggests  that  the  Commission  should  make  an  explicit  public  response  to  their 
recommendations. The present note serves exactly as that; a response to the recommendations made by the 
Panel.  The Commission  finds  these recommendations  valuable and timely,  since the debate  and  reflection 
on the Fifth  Framework Programme has started. 

88 

III. 

EVALUATION  PANEL 

Prof.  Dr. Rudolf  CASPER 
Prof.  Casper  participated  in the Panel  for Monitoring  1995 for the Biotechnology  Programme.  He was the 
Director  of the Institute  of Biochemistry  and Plant  Virology  of the Federal  Biological  Research  Centre  for 
Agriculture  and  Forestry,  Braunschweig,  Germany  until  1995.  This  institute  is  responsible  for  the 
evaluation and admission of field  experiments  with transgenic  organisms  in Germany. 

Dr Luke  Gregory  GEORGIOU 
Dr  Georgiou  is  the  Executive  Director  of  PREST  and has  experience  in  evaluations,  science  policy  and 
technological  innovation.  He  is a  founder  member  of  Commonwealth  Consultative  Group  on  Technology 
Management  and consultant  to the UK Technology  Policy. 

Dr Marti  LEISOLA 
Dr  Leisola  is  the  Technology  Director  of  process  technology  at  the  Technology  Development  Centre, 
which  coordinates  and finances  applied,  technical  and  industrial  R & D in Finland.  "He has experience  in 
evaluating  national  and  international  R &D  programmes. 

M.  Paskal  BYE 
Mr.  Byé,  Director  of Research  at INRA  (Institut  National  de Recherche  Agronomique)  participated  in the 
evaluation  of the  French  "Biotechnologies"  programme  which  covered  a  10 year  period  (1982-92).  This 
evaluation was organized  by the CNER  (Comité  National  d'Evaluation  de la Recherche). 

Prof.  Giampiero  RAMPONI 
Prof.  Ramponi  is a Director  of the Institute  of Biological  Chemistry  of the University  of Florence.  He is a 
member  of  the  progress  and  evaluation  for  the  group  on  Biotechnology  and  Bioinstrumentation  of  the 
CNR. 

Prof.  Luis  NAVARRO 
Prof.  L.  Navarro  has  been  the  manager  of  the  National  Programme  for  Agricultural  Research  in  the 
Interministerial  Commission  for  Science  and  Technology  (CICYT).  He  has  collaborated  with  FAO, the 
Hunan  Horticultural  Institute  in China  and the Agriculture  Ministry  of Cuba. 

Prof.  Gûnther KREIL 
As  well  as  his  academic  activities,  Prof.  Kreil  has  served  in  the  EMBL  Council,  the  EMBO  fellowship 
committee  at  the  Max  Planck  Institute  for  Biochemistry  (Munich)  and  for  Experimental  Medicine 
(Gôttingen). 

89 

1996 FIVE-YEAR ASSESSMENT 

BIOMEDIÇINE AND HEALTH 

91 

T.  FmirnvF. SUMMARY 

We have reviewed  the last five years of the European  Union Biomedicine and  Health  Research  Programme. 
The Review encompasses the three calls of Biomed  1, under the Third Framework  Programme1,  and the first 
call  of  Biomed  2  under  the  Fourth  Framework  Programme.  Our  assessment  of  the  Programme  and  our 
conclusions  and  recommendations  are based  on data provided  by the CEC,  various monitoring  reports  of the 
Biomed  1 and  Biomed  2 Programmes  and the analysis of data generated  by a questionnaire  which attracted  a 
response of 44%  from  Biomed  1 co-ordinators and 25% from  Biomed 2 co-ordinators. 
The  Research  Topics  of  Biomed  1  were:  Pharmaceuticals;  Occupational  and  Environmental  Health; 
Biomedical  Technology;  Public  Health/Health  Service  Research;  AIDS,  TB,  Infectious  Diseases;  Cancer 
Research;  Cardiovascular  Research;  Brain  Research;  Chronic  Diseases,  Ageing  and  Rare  Diseases;  Human 
Genome  Research;  Biomedical  Ethics. 
In  Biomed  2  the  topics  were  similar  with  the  addition  of  'Legal 
aspects and horizontal  activities'  including demonstration projects  as a separate topic. 
Both programmes  attracted  a large  number  of applicants;  1640 in the case  of  Biomed  1 of  which  403  (25%) 
proposals were  funded  and  1709 in the first  call of Biomed 2 of which 302 (18%) were funded.  In Biomed  1 
the  largest  number  of proposals  were  submitted  in the  Research  Areas  of  Brain  Research,  Cancer  Research, 
Cardiovascular  Research,  Chronic  and  Rare  Diseases  and  Ageing,  and  Human  Genome  Research.  The 
proportion  of  applications  to  Brain  Research,  Chronic  and  Rare  Diseases  and  Ageing  and  Public 
Health/Health  Service  Research  was  increased  in  Biomed  2  relative  to  Biomed  1.  In  Biomed  1,  concerted 
action (CA) was the main implementation mode, the only exception being Human Genome Research  in which 
the  CEC  did  fund  shared  cost  (SC)  proposals.  In  Biomed  2  the  predominant  mode  of  implementation  was 
SC. 
The total sum  requested  in Biomed  1 was MECU  824 and the amount  awarded  was MECU  134.  In Biomed 
2,  MECU  317 were  requested  and  MECU  146 were  awarded.  This  reflects  an  increase  from  16%  to 46% 
awarded  relative  to  that  requested  in  Biomed  1 compared  with  Biomed  2,  respectively.  The  average  EC 
contributions  per  contract  in Biomed  1 and  Biomed  2  (first  call),  respectively,  were  KECU  304  and  350  for 
CA and 585 and 614 for SC. 

We  conclude  that  the  Biomedicine  Programme  (Biomed  1  and  First  Call  of  Biomed  2)  has  achieved  the 
objective  of contributing  significantly  to the improvement  of medical  and health  research  and development  in 
Europe by facilitating  the establishment  of new collaborations  and/or  consolidating  and strengthening  existing 
collaborations.  The  Programme  has  satisfied  the  criteria  of  'European  added  value',  'European  dimension', 
and  'subsidiarity'  and  is characterised  by a multidisciplinary as well as multinational  approach  which makes it 
distinct from  Biomedical programmes  supported  by National  Research Bodies and  Industry. 
The  output  of  the  projects  supported  by  the  Programme,  as  assessed  by  the  analysis  of  Biomed  1,  would 
appear  to  be  qualitatively  and  quantitatively  equal  to  or  better  than  that  of  many  projects  supported  by  top 
National  Funding  Bodies.  The  shift  from  concertation  between  many  partners  in  Biomed  1  to  actual 
research  projects  between  smaller  groups  of  complementary  partners  in Biomed  2  is a  highly  positive  trend 
for  which the CEC  is to be commended. 
The  Biomed  Programme  has  also  established  links  with  other  Programmes  of  the  European  Union; 
integration  with  PECO  and  INCO3  has  promoted  participation  of  non-Member  States  from  Central  and 
Eastern  Europe. 
Notwithstanding  the success  of the  Programme,  we are  concerned  by  the  relatively  massive  oversubscription 
and therefore  wastage, due to the limited budget,  of a large amount of European talent and expertise. 

This 5-year  assessment  incorporates  the final evaluation of the Biomedicine and Health Programme under the Third  Framework  Programme 

Pays de l'Europe  centrale et orientale  (Central and Eastern European Countries) 

Specific  Programme for  International  Co-operation (in the 4th Framework  Programme) 

93 

The  method  of  selection  of  proposals  in  Biomed  2,  based  on  highly  efficient  and  effective  scientific  peer 
review,  is  a  marked  improvement  on  the  methods  of  selection  used  in  Biomed  1.  Nonetheless  further 
improvement  is  possible,  and  we  therefore  endorse  the  1995  Biomed  2  External  Monitoring  Report  which 
recommended  that  (i) unless there are exceptionally  compelling  circumstances,  proposals  should be  funded  in 
strict  rank  order  irrespective  of  whether  or  not  subareas  were  represented,  and  (ii)  that  the  funding  of 
proposals should be guided  by scientific  advice and follow  a stepwise or sliding scale proportional  to the rank 
order of the final score of the proposal. 
We are sensitive to the need for  the strategic, national  and political  factors  that may  of necessity  influence  the 
selection of proposals  in various areas.  A balance could be achieved between the funding  of  'response  mode' 
versus 'strategic top-down'  driven research  if some degree of science-led flexibility  (say  10%) was  introduced 
in determining  the size of the financial  envelopes that are set around the various Research  Topic Areas by the 
CEC  and  the  Member  States.  Nonetheless,  in order  to maintain  the apparent  high  quality  of the  outcome of 
the  Biomed  Programme,  and  to  ensure  equity  at  a  time  when  research  funds  are  exceedingly  scarce,  we 
recommend  that  irrespective  of  the  strategic  importance  of  any  given  area,  the  CEC  should  establish  a 
minimum  score  below  which  no  proposal  can  be  funded.  In  this  respect  we  commend  the  Commission  for 
implementing  adequate  monitoring  systems  in the form  of the Project  Review  Board  and  external  monitoring 
panels. 

Five  aspects  of  the  Programme  deserve  special  attention.  These  are:  (i)  the  relatively  small  proportion  of 
industrial  contractors;  (ii) the relatively  small number of proposals concerned  with research  on  'Illicit  Drugs'; 
(iii)  the  long  delay  between  the  receipt  of  proposals  and  the  transfer  of  funds  (median  of  12 to  16 months); 
(iv)  the  few  female  expert  evaluators  and  members  of  monitoring  panels;  (v)  the  shortage  of  CEC  Staff 
involved  in  the  Biomedicine  Programme.  Analysis  of  these  issues  and  recommendations  of  how  any 
deficiencies  might be rectified  are discussed  in the text and in the Conclusions and  Recommendations. 
We  highly  commend  the  EU  for  establishing  the  Biomedicine  Programme  which  has  a  major  impact  on 
European  Biomedical  Research,  and  will  have  an  important  positive  effect  on  the  prevention,  diagnosis, 
management  and  treatment  of  diseases  in  Europe.  The  Programme  provides  the  EU  with  an  incredibly 
powerful  network  and  database  of  unrivalled  expertise  in  Medicine  from  molecules  through  to  man  and 
Society,  which  in  addition  to  fostering  highly  productive  and  integrated  research  between  the  Members 
States,  serves  as  a vehicle  for  the  rapid  deployment  of  European  Task  forces  to  deal  with  strategic,  and/or 
national  issues as they  arise.  We very much hope that the Biomedicine  Programmes  of the Third  and  Fourth 
Framework  will  form  the  platform  for  a  similar  programme  in  the  Fifth  Framework,  and  that  such  a 
Programme  is  allocated  a  significantly  larger  budget  in  order  to  reduce  the  wastage  of  many  excellent 
initiatives which reflect  the Biomedical power of the Member States. 

Conclusion and  Recommendations 

(I) 

(ii) 

(iii) 

We  conclude  that  the  Biomed  Programme  (Biomed  1 and  First  Call  of  Biomed  2)  has  achieved  the 
objective  of  contributing  to  the  improvement  of  medical  and  health  research  and  development  in 
Europe  by  facilitating  the  establishment  of  new  collaborations  and/or  consolidating  and  strengthening 
existing  collaborations  in  Europe.  The  Programme  has  succeeded  in  satisfying  the  criteria  of 
'European  added  value', 
'subsidiarity'  and  is  characterised  by  a 
multidisciplinary as well as multinational  approach. 

'European  dimension',  and 

As a consequence,  the  research  supported  by  the  EU  Biomedicine  Programme  is quite  distinct  from 
the Biomedical programmes  supported  by National Research Bodies and  Industry. 

In addition to promoting  the formation  of  new  and major  European  Biomedical  consortia,  the  output 
of the projects  supported  by the Programme,  as assessed by  the analysis of Biomed  1, appear  to be of 
a  quality  and  quantity  equal  to or  greater  than  those  of many  projects  supported  by  National  Bodies. 
This  assertion  is based  on  (i)  the  positive  nature  of  the  new  links  forged  between  partners,  including 
industrial  partners,  (ii)  contributions  to  novel  methods  in health  care,  (iii)  the  relatively  high  number 

94 

(iv) 

(v) 

of patents that have been filed, granted  and licensed (i.e. contributions to wealth  creation),  and (iv) the 
high  impact  of  the  publications  produced  by  a  large  proportion  of  the  projects  supported  by  the 
Biomed  Programme. 

As  a consequence  of  CEC  policy  and  a substantial  increase  in budget,  there  was  a major  shift  from 
Concerted  Actions  and  Centralised  Facilities  in Biomed  1 to  Shared-Cost  projects  as  the  predominant 
implementation  mode  in  Biomed  2.  This  shift  from  concertation  between  many  partners  to  actual 
research  projects  between  a  smaller  group  of  complementary  partners  is  a  highly  positive  trend  for 
which the CEC  is to be commended. 

Notwithstanding  this positive trend,  we are concerned  by the fact  that the relatively  limited  budget of 
the  Programme  (albeit  increasing  over  the  years)  meant  that  only  a  minority  of  proposals  could  be 
funded  in Biomed  1, and  even  less  in Biomed 2.  That  is, the  level  of  oversubscription  was  relatively 
high  with  less  than  15% of  proposals  funded  (Biomed  2,  first  call)  in  Biomedical  Technology,  Brain 
Research  and  Cancer  Research,  and  a  mean  of  only  18%  of  proposals  funded  for  the  Programme 
overall.  Furthermore,  the  level  of  funding  for  each  selected  proposal  is variable,  and  in  some  cases 
might not allow achievement  of all the original objectives  that had  served  as a basis for  selection  of the 
projects.  The  apparent  positive  outcome  of  Biomed  1 (see  3  above)  suggests  that  for  CAs,  at  least, 
negotiations  between  the  CEC  and  the  contractors  led  to  a  formula  whereby  CEC  together  with 
National  or  other  (e.g.  Industrial)  contributions  ensured  the  success  of  the  research  proposed.  The 
outcome  of  Biomed  2,  in  which  SC  is  the  predominant  mode  of  implementation,  has  yet  to  be 
determined. 

(vi) 

The  Biomed  Programme  has  also  established  links  with  other  Programmes  of  the  European  Union; 
integration  with  PECO  and  INCO has  favoured  participation  of  non-Member  States  from  Central  and 
Eastern  Europe. 

(vii)  Monitoring  systems are  in place  in the form  of the Project  Review  Board  which  reviews annually  the 
progress  of  each project  and  has the power  to  implement  changes  or  terminate  unsatisfactory  projects 
on the basis of written  submissions  and,  if necessary,  site visits.  The CEC has also complied  with the 
EU  requirements  for  detailed  monitoring  of  the  Programme  as  a whole  of  which  the  present  and  the 
1995  External  Monitoring  Report  of  the  First  Call  of  Biomed  2  are  examples.  Nonetheless,  there  is 
room  for  improvement  in  that  monitoring  could  be  more  rigorous,  systematic  and  accessible.  We 
recommend  that  the  PRB should  continue  to monitor  the scientific  progress  of  the  Programme,  but  in 
addition  should  also monitor  the  impact  of the  funding  of the  Proposals  and  Areas  with  respect  to the 
Output  of  the  Programme.  The  administrative  monitoring  of  the  progress  of  contracts  should  be 
carried  out  by  dedicated  CEC  personnel  using  computerised  databases.  This  would  facilitate  more 
direct  and  rapid  evaluation  of  changes  which  are  made  during  the  Programme,  alert  the  CEC  to  the 
need for  additional changes and facilitate  greatly the work of the external monitoring panels. 

(viii)  The  method  of  selection  of proposals  in Biomed  2 is a marked  improvement  on the  methods  used  in 
Biomed  1.  We  strongly  commend  the  CEC  for  implementing  the  new  Biomed  2  selection  method 
which  entails  highly  efficient  and  effective  peer  review  and  at  the  same  time  secures  contractual 
confidentiality  and  secrecy.  The  latter  is  important  for  industrial  participation  and,  by  providing 
proponents  with  an  assurance  of  confidentiality,  may  have  played  a  significant  role  in  the  continued 
high response rate to the call for proposals  in Biomed 2. 

(ix)  We  endorse  the  1995  Biomed  2  External  Monitoring  Report  which  recommended  that  (a) unless 
there  are  exceptionally  compelling  circumstances,  proposals  should  be  funded  in  strict  rank  order 
irrespective  of  whether  or  not  subareas  are  represented,  and  (b)  that  the  funding  of  proposals  should 
be  guided  by  scientific  advice  and  follow  a stepwise  or  sliding  scale  proportional  to the  rank  order  of 
the  scores  of  the  proposals.  It  remains  to  be  seen  whether  these  two  principles  are  implemented  for 
the second  and third calls of Biomed 2. 

95 

(x) 

(xi) 

(xii) 

Notwithstanding  this recommendation  (ix) we are  sensitive  to the need  for  the  strategic,  national  and 
political  factors  that  may  of  necessity  influence  the  selection  of  proposals  in various  areas  depending 
upon  (a) the  special  needs  of  the  EU,  (b) the  occurrence  of  apparently  new  diseases  (e.g.  AIDS),  (c) 
the  recrudescence  of  known  diseases  (e.g.  tuberculosis),  (d)  the  emergence  or  prominence  of  known 
diseases  due  to  changes  in  demography  (e.g.  Alzheimer's  Type  Dementia  and  other  age-related 
disorders)  and/or  (e)  novel  developments  which  may  promote  health  care  or  wealth  creation.  We, 
therefore,  recommend  that  the  allocation  of  funds  to  Research  Areas  should  ensure  that  the  best 
science  is  funded  whenever  possible  and  at  the  same  time  allow  strategic  considerations  to  be  taken 
into  account.  A  balance  could  be  achieved  between  the  funding  of  'strategic  top-down'  versus 
'response  mode'  driven  research  if some degree  of science-led  flexibility  (say  10%) was introduced  in 
determining  the  size of the financial  envelopes  that are  set around  the various  Research  Topic  Areas. 
The  size  of the  financial  envelopes  would  still  be determined  by  the  CEC  and  the  Member  States  on 
the basis  of  strategic  considerations,  but  could  be  adjusted  by  the  Programme  Committee  in  response 
to the quality  of the proposals  in the different  Research  Areas  . This would  allow  a degree of  freedom 
in  shifting  funds  between  Areas  to  facilitate  changes  in  emphasis  or  success  in  European  Biomedical 
Research,  while  at  the  same  time  ensure  that  those  Biomedical  Areas  of  greatest  strategic  importance 
for  the Member  States would be represented  in any particular  call for  research proposals. 

However,  in  order  to maintain  the  apparent  high  quality  of  the  outcome  of  the  Biomed  Programme, 
and  to  ensure  equity  at  a  time  when  research  funds  are  exceedingly  scarce,  we  strongly  recommend 
that  irrespective  of  the  strategic  importance  of  any  given  area,  the  CEC  should  establish  a  minimum 
score  below  which  no  proposal  can  be  funded.  Using  the  present  Biomed  2  scoring  system  (0  to  4, 
where  4  is  excellent)  and  the  analysis  of  the  first  call  of  Biomed  2  (see  1995  External  Monitoring 
Report,  Figures  12 and  13), the minimum  average  score  for  the funding  of proposals  should  be  set at 
3,  which  is consonant with the CEC's own criteria of the borderline between  'excellent'  and  'good'. 

The proportion  of  industrial  participation,  and particularly  of  SMEs  (2.3% of  contractors)  is perhaps 
not as high as might have been wished  or expected.  The precise  reasons  for  this are not clear,  but one 
contributing  factor  may have been the duration of the time taken for the completion of the selection and 
funding  procedure  which  ranged  from  a  median  of  12  to  16  months.  Furthermore,  SMEs  may  be 
concerned  by  the  fact  that  although  "confidential",  proposals  are  nonetheless  exposed  to  extensive 
peer  review  and  are  not  assured  of  success  in the  competition.  Nonetheless,  all  of  the  16  companies 
who were partners  of  respondents to our  questionnaire  scored  the project  as  important  for  the business 
strategy  of  their  companies,  and  9  out  of  16  (56%)  scored  the  project  to  be  'very  important'  (top 
possible  score) for  their business strategy.  The CEC should take this as a commendation  from  Industry 
and as an encouragement  to explore ways of fostering  greater  industrial  participation. 
We  also  note that  the problem  of  relatively  few  SME participants  may be partly  offset  by  the  fact  that 
in  parallel  with  the  main  Biomed  Programme,  the  Commission,  in  1995,  initiated  a  scheme  for 
supporting  Exploratory  Awards  for  SMEs  as  well  as  Co-operative  Research  Projects  for  consortia  of 
4 SMEs.  The  impact  of  this  SME  Initiative  for  the Biomed  Programme  will need  to be  evaluated  at a 
later  date.  However,  we  recommend  that  the  CEC  ensures  that  the  advantages  for  SMEs  in 
participating  in  the  mainstream  Biomed  Programme  should  be  widely  advertised  (i.e.  given  greater 
visibility). 

(xiii)  The  Programme's  work  on  illicit  drugs  is not  as visible  as the  EU might  have  wished.  This  may  be 
due  to  the  fact  that  'Illicit  Drugs"  is  not  a  research  topic  in  its  own  right,  but  rather  a component  of 
several different  Research  Areas.  However,  there are significant  positive scientific  advantages derived 
from  the  integration  of  'illicit  drugs'  in  'Pharmaceuticals',  'Brain  research'  and  'Research  on  AIDS', 
for  example.  Given  the  high  level  of  oversubscription  of  the  Programme  and  the  relatively  small 
budget  for  Biomedicine,  establishment  of  'Illicit  Drugs'  as  a  separate  Research  Topic  might  be 
deleterious  to the Programme  as a whole unless  it was matched by  an appropriate  increase  in budget. 

96 

9. 

We  recommend  that  the present  system  is  retained  unless  special,  additional  sums  are  assigned  for 
specific  work on illicit drugs. 

(xiv)  We are concerned  by the relatively  long  delay  between  the  receipt  of  proposals  and the transfer  of 
funds  for  research,  especially  since  this  may  be  one  factor  which  might  deter  Industry  from 
participating  in the Programme.  However,  we are sensitive to the fact  that this delay  is due mainly to 
the  time  taken  by  the  negotiation  phase  which  given  the  small  number  of  CEC  Staff  and  the 
complexity  of  the  negotiations  is  of  necessity  quite  long.  This  delay  is  likely  to  be  reduced 
significantly  if the number  of CEC staff  was increased  from  18 to 28, the number  originally  allocated 
to the Programme  (see below). 

(xv) 

There  is  a  deficiency  in  affirmative  action  in  the  Programme  in  that  few  female  scientists  are 
appointed  as experts  for  the selection  process  and in the external  monitoring  teams.  We  recommend 
that the CEC takes positive steps to rectify  this situation and makes every effort  to become a  successful 
affirmative  action  employer. 

(xvi)  We have  all participated  in evaluation  and monitoring  processes  and thereby  witnessed  at  first  hand 
the excessive workload of the CEC Staff  involved  in the Biomedicine  Programme.  The CEC Staff,  as 
well  as having  to deal  with  Programme  Calls,  evaluations  and external  monitoring  Committees,  also 
have  to  manage  effectively  and efficiently  700, and by the end of  1997, about  1200 contracts.  The 
contracts  involve  complex  scientific  and financial  issues  in  several  to many  dispirit  Member  States. 
We believe that the excessive workload,  which is due in part to the relatively  small number  of Staff,  is 
not  consistent  with  either  good  and  humane  work  practice  or  efficient  and  effective  Programme 
Management.  We therefore  strongly  recommend  that  because  of  its central  importance  to  European 
Biomedical  Science,  Health  Care  and  Wealth  Creation  and  because  of  its  obvious  popularity,  the 
Biomedicine  Programme  should be awarded the 28 staff  which it was originally  allocated. 

(xvii) 

Notwithstanding  some  of  the  concerns  mentioned  above,  we  highly  commend  the  EU  for 
establishing  the  Biomedicine  Programme  which  has  already  had  a  major  impact  on  European 
Biomedical  Research,  and  will  make  significant  contributions  to  the  prevention,  diagnosis, 
management  and treatment of diseases in Europe.  Furthermore,  although the proportion  of Industrial 
Partners  may not be as large  as originally  hoped  for,  the obvious  importance  of the Programme  for 
industrial  contractors  suggests  that  with  further  targeting  of  the  private  sector,  the  Biomedicine 
Programme  could also play a significant  role in wealth  creation. 
Our  independent  analysis  has  shown  that  the  Biomedicine  Programme  has  a  high  European 
Dimension  and  Added  Value  and,  that  the  output  of  the  Programme  in  terms  of  new  European 
initiatives, health care developments, potential wealth creation and quality  of publications  is generally 
excellent.  The  Programme  provides  the  EU  with  an  incredibly  powerful  network  and  database  of 
unrivalled  expertise  in Medicine  from  molecules  through  to man and  Society,  which  in  addition  to 
fostering  highly  productive  and integrated  research  between  the Members  States,  serves  as a vehicle 
for  the  rapid  deployment  of  European  Task  forces  to  deal  with  strategic,  and/or  national  issues  or 
crises  as they  arise.  The Programme  can already justly  and proudly  claim  major  successes  such as 
the discovery  of a new variant  of Creutzfeldt-Jakob  Disease,  the improvement  of the survival  rate to_ 
70%  of a previously  fatal  disorder  (severe  combined  immunodeficiency)  and many  others  which are 
exemplified  in our Report.  The high  standard,  efficiency  and efficacy  of the Programme  depends in 
large measure on the work of the CEC staff  who deserve  special commendation.  We have  identified 
several  areas in which technical  improvements can still be made, but the Programme provides the EU 
with a model for other research  initiatives pivotal for the growth and development  of Europe. 
The  Biomedicine  Programme  offers  an  important  mechanism  by  means  of  which  individual 
biomedical  talents  in Member  States can be harnessed  for the greater  good  of the European  Union. 
We  very  much  hope  that  the  Biomedicine  Programmes  of  the  Third  and  Fourth  Framework 
Programmes  will form  the platform  for a similar programme  in the Fifth  Framework,  and that such a 

97 

Programme  is allocated  a significantly  larger budget  in order  to reduce the wastage  of many  excellent 
initiatives which reflect  the Biomedical power of the Member  States. 

98 

IT. 

COMMISSION'S  RESPONSE 

The  Commission  notes  the  Panel's  comment  that  both  3rd  and  4th  Framework  Programme  activities  under 
the  BIOMED  Programme  have  achieved  the  objectives  to  contribute  to  the  improvement  of  medical  and 
health  research  in  Europe  and  satisfied  the  criterias  of  European  added  value,  European  dimension  and 
subsidiarity.  The Commission  notes the clear distinction the evaluators observe between  the BIOMED  funded 
and  nationally  funded  research  which  is in strict  compliance  with  the principle  of  subsidiarity.  Regarding  the 
output  of  the  projects  supported  by  the  Programme,  the  Commission  notes  the  Panel's  observation  that  it 
appears to be  of  a quality  and  quantity  equal to or  greater  than those  of many  projects  supported  by  National 
Bodies. 

The  trend  towards  more  high  quality  Shared  Cost  Actions  being  proposed  and  selected  for 
is 
recognized  by  the  Commission.  This  is  taken  as  an  indication  of  the  increasing  demand  of  actual  research 
funding  on  the  European  level.  Although  the  overall  success  rate  of  the programme,  in terms  of  number  of 
projects,  is similar  to NIH (USA) and  other  funding  bodies, the Commission  shares  the concern  of the Panel 
regarding  the  oversubscription. 

funding 

Regarding  the  recommendation  on  the  allocation  of  funds  to  the  various  research  areas,  the  Commission 
notes that  indicative  budgets  for  the  Programme  established  by  the  Council  can  be  and  have  been  amended, 
depending  upon  research  priorities,  as  a  specific  response  to  a  call  and  following  a  positive  opinion  of  the 
Programme  Committee.  The  Commission  also  notes  that  a high  priority  is set  by  the  Panel  for  research  on 
illicit drugs and that this area  is seen as an integral part of the biomedical  research  activities.  This  is indeed  in 
compliance  with the recent Council positions on the matter. 

The Commission  is pleased  to receive  the comment  on the  improvements  regarding  the selection  procedures, 
which  are  now  described  as  efficient,  effective  and  confidential,  and  it  will  continue  to  improve  evaluation 
procedures.  The  recomendation  that a minimum  score  should  be established  below  which  no proposal  can be 
funded  has  already  been  implemented  in  the  second  call.  The  analysis  of  the  second  call  shows  marked 
increase  in  the  industrial  participation  however,  health  care  funding  agencies  (undertakings,  hospitals)  are  a 
major  participating  force  in the BIOMED programme. 

In  terms  of  positive  action  towards  having  more  female  scientists  appointed  to  the  selection  procedures,  it 
must be  noted  that BIOMED  staff  is 50%  male and  50% female.  The Commission  has only  limited  choice  in 
the academic scientific  evaluators.  The ratio of male to female  in the names proposed  by Member  States 
is 9:1. 

The  Project  Review  Board  (PRB)  is the  main  instrument  relating  to the  ongoing  monitoring  of  the  scientific 
progress of the Programme,  and the Panel's recommendation to enhance the role of this body  is noted. 

The  small  number  of  scientific  staff  involved  in the  Programme  administration  at present,  recognized  by  the 
Panel,  is  a  very  serious  problem  and  efforts  are  being  made  to  find  well-qualified  staff  in  the  future. 

99 

III. 

EVALUATION  PANEL 

Prof.  S.  AYMÉ 
Director  of  Research  at  INSERM.  A national  expert  in Human  Genetics,  Epidemiology  and  Public  Health 
issues  for  genetic  diseases  (INSERM  and  CNRS)  and  on  on  rare  diseases  (Ministry  of  Health).  At  the 
European  level,  an  expert  for  the  Human  Genome  Project.  President  of  the  European  Society  of  Human 
Genetics.  In  US,  Advisory  board  member  of  the  Genome  Database.  Member  of  several  editorial  boards. 
Member  of the  EUROCAT  association. 

Prof.  U.  FAUST 
Former  Head  of  the  Instutute  of  Biomedical  Technology  in  Stuttgart  and 
long  term  member  of  the 
BIOMED  Project  Rewiev  Board.  Wide  specialities  on  the  biomedical  engineering  research,  specially  on 
medical  imaging,  image  processing  and  sensor  technology.  Several  advisory  tasks  for  industrial  R&D 
projects. 

Prof.  G.  FINK  (Chairman  and  Rapporteur) 
Director  of  the  Medical  Research  Council's  Brain  Metabolism  Unit,  elected  Honorary  Professor  in  the 
University  of  Edinburgh  in  1984.  Professor  Fink  is  responsible  for  setting  the  scientific  strategy  of  the 
Unit,  deployment  of  the  Unit's  Staff  and  management  of  the  Unit's  core  budget.  His  personal  expertise  is 
in  neuroendocrinology,  neuropharmacology  and  neuroendocrine  molecular  biology.  His  opinion  is  widely 
sought  as  a  referee  in the  area  of  Neuroscience  on both  sides  of  the  Atlantic.  He  served  on  the  Wellcome 
Trust  Panel  for  Mental  Health  and  Neuroscience  between  1984-1989.  His  refereeing  has  involved  the 
evaluation  of  several  major  institutions.  He  is  on  the  Editorial  Board  of  several  major  journals.  President 
of  the  European  Neuroendocrine  Association  (1991-1995).  Member  of  Council  of  the  European 
Neuroscience  Association. 

Prof.  A.  GRAVANIS 
Regular  reviewer  of  research  proposals  funded  from  the  Greek  General  Secretariat  of  Research  and 
Technology  and  the  Board  of  Biomedical  Research,  Greek  Ministry  of  Health.  Recognized  expert  on 
molecular  pharmacology  of  hormone-dependent  tumors.  Autocrine/paracrine  interactions  of  neuropeptides 
and  steroid  hormones  in  the  tumors  of  the  Repro-ductive  System.  Molecular  mechanisms  of  Oncogene 
regulation  in  cell  proliferation/differentiation.  Pharmacological  evaluation  of  new  hormone  antagonists  in 
the hormonal  therapy  of  cancer. 

Prof. J.M.  MARTIN-MORENO 
He  is  a  Visiting  Fellow  of  the  Harvard  School  of  Public  Health,  an  Associate  Researcher  of  the  Milan-
based  European  Institute  of  Oncology  and  a Professor  in the  department  of  Epidemiology  and  Biostatistics 
at the  National  School  of Public  Heath  in Madrid,  Spain,  where he is now the  Director. 
implementation  and  evaluation  of  research  projects  at  a  national  and 
His  experience  ranges  from 
international  level  to  previous  chairmanship  of  the  Discipline  Oriented  Group  on  Epidemiology  and 
Clinical  Trials  within the  BIOMED  I programme,  and current  membership  of the  Project  Review  Board  of 
the  European  Commission's  BIOMED  programme.  He  is  interested  in  epidemiological  methods,  meta 
analysis  and,  particularly,  in  the  nutritional  epidemiology  of  cancer,  a  subject  on  which  he  has  published 
numerous  papers  at home  and  abroad. 

100 

Prof. L. NOTARANGELO 

Head, Laboratory  Unit of Molecular Immunology, Department of Pediatrics, University of Brescia. Senior 
Clinical  Investigator,  Hematology-Immunology  and  Bone  Marrow  Transplantation  Unit,  Dept  of 
Pediatrics, Univeristy of Brescia. Permanent Member, WHO Advisory Committee on 
Primary  Immunodeficiencies.Member  of the Editorial  Board  of several  Italian  and  International  Pediatric 
Journals.  Member  of the Evaluation Committee,  Italian National Health Institute Research  Grants (1992). 
Member of the External Monitoring Report Committee, Biomed 2 Programme (1995). 
His recent contributions include: 
of  X-linked 
immunodeficiency  with  hyper-IgM,  identification  of the gene  responsible  for  autosomal  recessive  Severe 
Combined  Immune Deficiency,characterization  of the molecular basis of X-linked  thrombocytopenia,  first 
European  clinical  gene  therapy  trial  for  adenosine  deaminase  deficiency.  Scientific  Secretary,  European 
Society for Immune Deficiencies. 

the  molecular 

characterization 

basis 

of 

101 

10. 

1996 FIVE-YEAR ASSESSMENT 

AGRICULTURE AND FISHERIES, 

INCLUDING AGRO-INDUSTRY 

103 

10. 

ï. 

EXECUTIVE  SUMMARY 

The  panel  was  asked  to  assess  research  programmes  in  agriculture,  fisheries,  forestry  and  agro-industry 
for  the five  years  1991-96.  This period  covered  all of the Third,  and parts  of the Second  and  Fourth 
Framework  Programmes.  These  included  over  850  projects,  with  approximatel  675  MECU  of  EU 
funding. 

The  changing  balance  of  the  programme  shows  a  well  reasoned  evolution.  There  has  been  an  increasing 
emphasis  on  forestry  and  fisheries,  as well  as on  food,  nutrition  and  environment.  Most  of  this  change  has 
been  policy  driven.  The  panel  felt,  however,  that  the  research  programme  should  lead  policy,  as  well  as 
follow  it.  This means  that  there  should  be a more  explicit  place  in the programme  for  the  pursuit  of  longer 
term goals. 

The  panel  undertook  a  direct  assessment  of  a  randomly  selected  sample  of  15%  of  the  projects  in  AIR. 
Projects  were  scored  for  progress  in  relation  to  scientific  goals,  potential  impact,  effectiveness  of 
dissemination,  and  linkages  created.  For  all  four  criteria,  at  least  82%  of  projects  were  judged  to  have 
performed  "in  line  with  expectation"  or  better.  Based  on  the  detailed  results  of  this  assessment,  and  the 
evidence  from  existing  reports  on  other  programmes,  the  panel  concluded  that  the  programme  as  a  whole 
is very  successfully  meeting  its  objectives. 

The  replacement  of  the  separate  sectoral  programmes  of  FP2  with  the  integrated  AIR  programme  of  FP3, 
and  FAIR  in  FP4  has  worked  well.  Co-ordination  between  the  three  DGs  concerned  appears  good. 
Linkages  to  eight  parallel  programmes  are  important,  but  there  is  little  information  on  how  effective  they 
are. 

Despite  some  excellent  examples  (e.g.  AIR  project  catalogue)  the  overall  information  system  on  EU 
programmes  is not well adapted  to facilitate  analysis and  management. 

The  information  base  from  which  to  analyse  and  co-ordinate  linkages  between  the  EU  programmes  and 
national  programmes  is even  less  satisfactory. 

Broadly  speaking,  the  programmes  were  very  well  managed.  This  judgement  was  supported  by  the 
opinions  of  378  project  co-ordinators  who  responded  to  a  survey.  The  panel  has  made  a  number  of 
suggestions  (see Recommendations)  for  improvement  in the management  procedures. 

One  area  requiring  additional  attention  is  dissemination.  Communication  of  results  to  the  scientific 
community  appears  to  be  excellent,  but  communication  to  intermediate  and  end-users  could  be  improved. 
The  FLAIR  FLOW  model  was judged  to be a good  example. 

Recommendations : 

1.  The  research  agenda  should  provide  for  more work on  longer  term  issues, beyond  the  immediate  policy 

objectives,  and  on systems  and themes  which may  not be economic  today. 

2.  Research  across  the  spectrum  of  activities  encompassed  in  the  term  "sustainability"  should  be  more 
strongly  encouraged.  This  should  be  done  by  delineation  of  areas  where  increased  work  is  seen  to  be 
necessary,  as well  as by  including  a sustainability  criterion  in the guidelines  on projects. 

3.  There  should  be  increased  emphasis  on  research  which  responds  to  the  growing  consumer  demand  for 

information  on and  confidence  in the  integrity  of the food  chain. 

4.  The  agenda  should  encourage  a proportion  of projects  which  reach  out  to the  leading  edge  of  scientific 

development. 

5.  Linkages  to the environmental  program  need  to be  increased. 
6.  Linkages  to  national  programmes  need  to  be  improved,  initially  by  developing  an  effective  common 

database. 

105 

10. 

7.  The present  procedure  for  project  submission  could  be  improved  by  providing  for  submission  of  outline 

proposals  in a pre-screening  phase. 

8.  On completion,  each project  should  be  subject  to an  independent  review. 
9.  More  effective  dissemination  of  results  is  needed.  This  applies  particularly  beyond  the  level  of 
scientific  publication,  which  seems  adequate.  The  FLAIR  FLOW  example  could  be  a  model  in  some 
areas.  Better  promotion  of  existing  mechanisms  (Mobility,  Dissemination)  in  connection  with  the 
research  programme  might  also be  useful. 

10.  Greater  involvement  of  industrial  partners,  particularly  SMEs,  should be  encouraged. 
11.  Concerted  Action type  activities  should  be  increased. 
12.  The  management  of  the  programme  by  the  Commission  Services  needs  to  be  strengthened,  in  some 
areas  by  the  deployment  of  additional  staff,  and  particularly  by  the  investment  of  more  resources  in 
an efficient  Management  Information  System. 

106 

II. 

COMMISSION'S  RESPONSE 

The  Commission  welcomes  the  external  evaluation  report  concerning  Community  research  programmes  in 
the  field  of  agriculture,  fisheries  and  agro-industry  for  the  five-year  period  1991-1996.  The  report 
provides  a good  insight  of the evolution  of successive  programmes  and  the Commission  is pleased  with  the 
overall  positive  opinion.  The  recommendations  made  by  the panel  will  be  considered  when  implementing 
the  remaining  part  of  the  Fourth  Framework  Programme  and  when  preparing  the  Fifth  Framework 
Programme. 

The  objectives  of  Community  research  programmes  in the fields  of  agriculture,  fisheries  and  agro-industry 
have been manyfold  but the most  crucial  aims  include:  to promote  and  harmonize  trans-European  research 
and  build  a  strong  scientific  basis  in  Europe;  to  strengthen  the  competitiveness  of  European  industries  in 
these  sectors,  and  to  provide  support  to  Community  policies.  The  latter  objective  does  not  necessarily 
imply  immediate  needs,  for  which  other  sources  of  funding  often  exist,  but  mostly  involve  long-term 
policy  objectives  (e.g.  rural  development,  sustainable  exploitation  of  fisheries  resources).  Similarly,  many 
basic  research  areas  such  as plant  and  animal  health,  generic  science  for  nutritious  foods  or  the  biology  of 
new  aquaculture  species  have  been  included  in previous  and  current  EU  programmes.  Furthermore,  many 
cross-links  have  been  established  with  other  specific  EU  programmes  involving  some  more  basic  research 
aspects  related  to  the  Biotechnology,  Biomedicine,  Energy,  Materials  and  Industrial  Technologies, 
Environment  or  Training  and  Mobility  programmes.  In  all  areas  of  the  previous  workprogrammes, 
scientific  and  technological  innovation  stands as the most  important  criterion  for  selection  and  it  is  strongly 
underlined  in calls  for  tender. 

Sustainability  issues  have  received  increasing  attention  during  the  five-year  period  and  are  currently 
addressed  in  many  areas  of  the  FAIR  workprogramme.  The  sustainability  criteria  suggested  in  the  report 
will  be  investigated.  Linkages  to  the  environmental  programme  currently  exist  through  interservice 
consultations  and  meetings  for  specific  areas  of  the  workprogramme.  In  the  context  of  the  Fifth 
Framework  Programme,  research  in the  fields  of  life  sciences  and  environmental  sciences  are  likely  to  be 
more  closely  related. 

Several  attempts  have  been  made  to  overcome  the  difficulties  and  limitations  associated  with  linkages  to 
national  programmes.  A  directory  of  fisheries  and  aquaculture  research  organizations  including  their 
research  activities  has  been  successfully  produced  and  an  inventory  of  agricultural  research  projects  has 
been  supported  in  an  initiative  named  AGREP  which  is  an  information  system  for  ongoing,  publicly 
funded  research  projects  currently  implemented  in  11  EU  Member  States.  Further  to  the  Commission's 
'Achieving  coordination  through  cooperation',  a  CREST  ad  hoc  group  has  been 
communication 
established  in  1996  for  facilitating  the  exchange  of  information  on  national  RTD  activities.  During  the 
first  year  of  activity,  the  group  has  compiled  a  large  amount  of  relevant  national  information  and  has 
produced  a  valuable  report.  Concertation  activities  have  continuously  been  strongly  encouraged  and 
priority  has always  been  given to funding  of concerted  action proposals  at the selection  stage.  The  concept 
of  thematic  network  has  been  promoted  in  the  FAIR  programme  to  reinforce  concertation  activities. 
Efforts  will  be  pursued  to  stimulate  high  quality  concerted  actions,  particularly  in  areas  of  strategic 
importance  for  Europe. 

The  level  of  industrial  participation  in  projects  strongly  depends  on  the  research  areas  involved.  Agro-
industrial  projects  involve  almost  always  at  least  one  industrial  partner  and  show  an  increasing  industrial 
In  primary 
participation  during  the  five-year  period  with  a  current  involvement  higher  than  25  %. 
production  areas,  industries  are  usually  not  technology  oriented  and therefore  have  not  been  very  active  in 
European  research  programmes.  The  technology  stimulation  measures  for  SMEs  which  have  been  put  in 
place  in  FAIR  have  attracted  a  growing  number  of  proposals  and  have  proved  to  be  an  efficient  tool  for 
raising  SME participation  particularly  in the food  technology  domain. 

107 

10. 

Dissemination  of  results  is  a  vital  component  of  the  Community  research  programmes.  While 
dissemination  to  the  scientific  community  has  proved  to  be  very  effective  through  scientific  journals  and 
conferences,  dissemination  to  end-users  requires  more  careful  attention.  Specific  networks  have  been 
established  in  the  food  and  non-food  sectors,  whereas  dissemination  in  other  domains  of  the  programme 
relies  more  on  national  initiatives  and  dissemination  plans  implemented  in each  project.  Dissemination  of 
results  to SMEs  will be ensured  through  the recently  established  FAIR  network  of  focal  points.  Consumer 
trends  in general  and  the  growing  consumer  demand  for  information  on  and  confidence  in  the  integrity  of 
the  food  chain  in particular  have  been  increasingly  addressed  in  successive  programmes  and  are  currently 
specifically  identified  in the  context  of  accompanying  measures  on ethical,  legal  and  social  aspects.  Indeed 
the role which  food  plays  in human  health  has been  given  increasing  importance  starting  from  a small  base 
in FLAIR,  continuing  in AIR  and  representing  now  a significant  area  in  FAIR. 

Programme  management  improvements  are  continuously  sought.  While  informal  pre-screening  has  been 
very  active  for  guiding  applicants  to  the  most  appropriate  sources  of  funding,  it  is  felt  that  a  formal  pre-
screening  step  could  create  even  bigger  disappointments  among  proposers  and  that  targeting  calls  for 
specific  areas  could  be  a  more  effective  mechanism  to  combat  oversubscription.  Various  ways  for 
strengthening  project  monitoring  and  assessment  after  completion  are  also  being  considered  including  the 
information  management  system  is 
assistance  of  external  experts.  Finally  the  need  for  an  efficient 
recognized  by  the  services.  New  information  systems  have  been  recently  developed  (e.g.  AMPERE)  but 
further  improvement  are  necessary  to  allow  harmonized  and  up-to-date  information  management.  This 
issue could  also be  resolved  by  increasing  human  resources  at the Commission  level. 

In  conclusion,  most  of  the  recommendations  made  by  the  panel  prove  to  be  consistent  with  the  first 
orientations  for  the  Fifth  Framework  Programme,  proposed  by  the  Commission  in  'Inventing  tomorrow'. 
The  Commission  strongly  believes  that  this  independent  evaluation  of  EU  RTD  activities  will  thus  be  very 
valuable  for  orienting  further  implementation  of  the  Fourth  Framework  Programme  and  for  preparing  the 
Fifth  Framework  Programme. 

108 

m. 

EVALUATION  PANEL 

Prof.  Stefano  Cataudella,  Università  degli Studi di Tor Vergara  (IT) 
Professor  Cataudella,  Associated  Professor  in the  University  Tor Vergara  (Rome) has a wide  experience  in 
research,  especially  in  aquaculture-environment  relationships  and  in  genetic  characterization  of  reared 
population  of  fish.  Concerning  academic  experience  he  has  been  involved  in  different  university 
departments  since  1973  and  co-operated  with  international  programmes  for  development.  He  also 
participated  in the evaluation  of  FAR  projects. 

Prof.  Patrick  Cunningham,  Trinity  College Dublin  (IE), Panel  Chairman 
Professor  Cunningham  was Director  of the Animal  Production  and  Health  Division  at the  FAO until  1993. 
Since  then,  he  returned  to  his  position  as  Professor  of  Animal  Genetics  at  Trinity  College  and  is  very 
active  in  various  international  advisory  committees.  His  research  work  has  been  centred  on  quantitative 
genetic  theory  and  on the  efficiency  of livestock  improvement  programmes. 

Mr Francisco Javier Pereiro Mufioz,  Instituto Espanol  de Oceanografia  (ES) 
Mr  Pereiro  Mufioz  has  a  twenty-year  experience  in  fisheries  research  ,  especially  in  demersal  species  in 
European  waters.  He  is the  head  of  the  IEO  Fisheries  Research  Programmes  in  European  waters  and  has 
been  member  of  the  Scientific  and  Technical  Committee  for  Fisheries  (STCF)  of  the  EC.  He  is  Spanish 
delegate  in  some  committees  of  the  International  Council  for  the  Exploration  of  the  Sea  (ICES)  and  in  the 
ACFM  (Advisory  Committee  for  Fisheries  Management). 

Dr Daniel  Richard-Molard,  Institut National  de la Recherche Agronomique  (FR) 
Dr  Richard-Molard  is currently  the  Head  of  the  Protein  and  Carbohydrate  Technology  department  as  well 
as  the  Fruit  and  Vegetable  Biotechnology  department  at  INRA.  His  research  work  is  focused  on  the 
transformation  technologies  of  cereals  for  food  and  non-food  purposes  and  involves  many  cooperation 
programmes  with  industry. 

Prof. Jorma  Sundquist,  The Finnish Pulp and Paper Research Institute  (FI) 
Professor  Sundquist  was  the  Research  Director  of  the  Paper  Science  Centre  at  the  Finnish  Pulp  and 
Research  Institute.  His  research  interests  cover  polymer  science,  wood  and  pulping  chemistry  as  well  as 
chemical  textile  technology.  He also participated  in the evaluation  of  FAIR  projects. 

Prof.  S.  Athanasios  Tsaftaris,  Agricultural  University  of Thessaloniki  (GR) 
Professor  Tsaftaris's  research  interests  include  many  different  aspects  of  plants,  particularly  biochemistry, 
physiology,  genetics,  breeding  and  biotechnology.  Professor  Tsaftaris  is currently  serving  in  the  National 
Research  Council  of Greece  and  is a member  of several  international  scientific  advisory  boards. 

Prof.  Marcel  Vanbelle,  Université  Catholique  de Louvain  (BE) 
Professor  Vanbelle's  area  of  expertise  covers  animal  and  human  nutrition.  He  has  been  a  member  of  the 
Belgian  Council  of  Human  Nutrition  since  1990,  has published  more  than  450  papers  and  is  a member  of 
several  international  scientific  advisory  boards. 

Prof.  Heinrich Wohlmeyer,  Agricultural  University  of Vienna  (AT) 
Professor  Wohlmeyer  is  an  agricultural  scientist  with  international  training  in  economics  and  commodity 
agreement.  An  industrial  experience  in  agro-industry  preceded  his  current  research  position  at  the 
University.  His  research  interests  involve  natural  resource  management  and  environmental  economics. 

109 

Il 

1996 FIVE-YEAR ASSESSMENT 

NON NUCLEAR ENERGY 

1 1! 

11. 

I. 

EXECUTIVE SUMMARY 

1.1  Background 

This is a self-contained  summary of the report of the Assessment Panel on the Non-Nuclear Energy (NNE) 
RTD&D activities  of the  EC for  the period  1991-1996.  The summary  forms  an integral part  of  the main 
report of the Panel. The Assessment was carried  out by a panel of independent  experts, whose names and 
Terms of Reference  are annexed to the main report. These RTD&D activities took place under the JOULE 
Programme  within  the  Third  Framework  Programme  and  under  the  THERMIE  Programme,  and  under 
their successor, the JOULE-THERMIE Programme, within the Fourth Framework Programme. 

As no final  evaluation of these programmes was carried out under the Third  Framework  Programme, this 
Assessment  subsumes  the  evaluation  of  the  activities  under  the  Third  Framework  Programme,  in 
accordance with the Commission's Communication to Council COM(96) 220 (final) of 22 May,  1996. 

1.2  The NNE  RTD&D  activities 

The  oil  crises  of  the  seventies  spurred  the  EC  interest  in  energy  research  and  demonstration,  with  the 
research  element  being  managed  by  DG  XII  and  the  demonstration  element  by  DG  XVII.  The  energy 
research  programme  became  known  as the  "JOULE"  Programme  in  1989 under  the  Second  Framework 
Programme,  and  the  demonstration  element  as the  "THERMIE"  Programme,  which  was  not  part  of  the 
Framework  Programme.  The  two  elements  were  amalgamated  to  form  the  "JOULE-THERMIE" 
Programme  in  1994, under the Fourth Framework Programme. The two elements continue to be  formally 
administered  and managed by DG XII and DG XVII, respectively,  and to have two separate  management 
committees. 

The objectives  of the two programmes and their successor are described in Section 3.1 and in Annexe 2 of 
the main report.  As a generalisation, project  co-ordinators  in the JOULE Programme tend to be located in 
universities  and  research  organisations,  whereas  those  in  the  THERMIE  Programme  are  industrial 
enterprises or industrial research institutions. While individual project budgets under THERMIE tend to be 
larger than those under JOULE, the overall programme budgets are comparable. 

The technical activities within the NNE programmes cover a wide domain including Renewables Energies, 
the  Rational  Use  of  Energy,  and  Fossil  Fuels.  In  addition  to  research  and  demonstration  actions,  the 
domain includes associated measures such as supporting and dissemination actions, and strategic studies. 

1.3 Structure of the main report 

The  main  report  has  been  structured  in  accordance  with  the  format  suggested  by  CREST  for  the 
assessment of all specific  RTD&D programmes. 

- Section 1 presents a self-contained  summary of the main report. 
- Section 2 is an introduction to the report and to the methodology used. 
- Section 3 provides a detailed critique of the programmes. 
- The Assessment Panel's conclusions are presented in Section 4. 
- The Panel's recommendations are presented in Section 5. 
- Supplementary  information  is contained in the Annexes. 

1.4 Methodology of the Assessment 

This assessment was carried out at the programme level, not at the project  level. A questionnaire was sent 
by  the  JOULE  and  THERMIE  managements  to  a  random  sample  of  340  project  co-ordinators.  The 
questionnaires  were  returned  directly  to  the  rapporteur  over  a  period  of  months.  Analyses  of  these 

113 

11. 

questionnaires  are  presented  in  Annexes  3  and  4.  Background  information  on  the  programmes  was 
provided  by  the  Commission  services  to  this  Panel  at  the  beginning  of,  and  during,  the  Assessment.  The 
Assessment  panel  designed  a  questionnaire,  which  was  distributed,  with  the  permission  and  co-operation 
of  the  Programme  Directors,  to  programme  managers  and  staff  responsible  for  the  JOULE  and 
THERMIE  elements  of  the  Programme.  Responses  were  returned  anonymously  to the  rapporteur,  and  are 
presented  in summary  form  in Annexe  5.  The  Assessment  Panel  interviewed  the Directors  of  the  JOULE 
and THERMIE  programmes  with some of their  staff. 

The  Assessment  Panel  first  met  in  Brussels  on  2nd  September,  1996  and  on  five  occasions  between  then 
and 20th  December,  1996. 

1.5  Assessment  of  the  NNE  RTD&D  activities 

By  far  the  greater  part  of  the  EU's  energy  supply  still  comes  from  non-nuclear  sources  and  will  continue 
to  do  so  for  the  foreseeable  future.  The  programme's  concern  with  non-nuclear  energy  is  therefore  still 
justified.  There  remain  concerns  about  security  of  energy  supply,  environmental  impacts  and  other  issues. 
The objectives  of the Community's  actions  in the domain of NNE RTD&D  were to: 

-  Increase  the  long-term  security  of energy  supply 
- Reduce  consumption  and  imports  of  energy 
- Alleviate  environmental  problems  related  to energy  production  and  consumption. 

These  objectives  were  to  be  served  through  carrying  out  RTD&D  on  the  rational  use  of  energy  and 
exploitation  of  renewable  and  new  indigenous  sources  of  energy.  They  were  reinforced  and  broadened  to 
take  account  of  Union  policies  and  of  socio-economic  and  political  changes.  The  initial  objectives  of  the 
programmes  are  still  valid  against  evolving  scientific,  technical,  industrial  and  socio-economic  conditions. 
In  the  Assessment  Panel's  opinion  they  will  remain  largely  valid  for  the  Fifth  Framework  Programme, 
taking  account  of the availability  of energy  technologies  and  of their  slow market  penetration. 

Selection  criteria,  procedures,  objectives  and  scope  of  the  activities  as  laid  down  in  Council  Directives 
have  been  respected  by  the  Commission.  However,  it has  proved  difficult  to  adopt  specific  and  verifiable 
Programme  objectives  against  which  performance  can  be  evaluated.  This  is  a  common  problem  in 
research. 
In  the  Panel's  opinion,  the  NNE  activities  have  been  managed  in  a  cost-effective  manner.  The  staff 
resources  allocated  to management  are,  if anything,  somewhat  inadequate  resulting  in overload  and  delays. 
Nonetheless,  the  management  of  the  activities  is generally  satisfactory  and  efficient.  Information  provided 
to  proposers  has  improved  in  quality  in  recent  years  and  the  project  selection  process  in  the  JOULE 
element  is  transparent.  Although  the  NNE  activities  fall  within  one  programme,  the  JOULE-THERMIE 
Programme,  there  are  in fact  two management  committees  and there  are  legal  separations  between  the  two 
elements.  An  external  monitoring  process  was  introduced  in  1995  and  has proved  useful  to this  Panel  and 
to the Commission  services. 

The  management  of  the  Programme  has  appropriate  modes  of  action  open  to  it  and  adequate  flexibility  in 
operation.  However,  this freedom  has not always been utilised  to best  advantage. 

The  Assessment  Panel  found  it difficult  to judge  the  effectiveness  of the Programme  or  the  achievement  of 
objectives.  This  is  a problem  common  to  all  research  activities  which  have  broadly-stated  objectives,  and 
is  compounded  by  the  fact  that  national  governments  and  private  organisations  also  carry  out  NNE 
RTD&D,  making  the Commission's  achievements  difficult  to isolate.  However,  there  are  areas  in all  three 
main  sectors  of the  Programme  (Rational  Use  of  Energy,  Renewable  Energies  and  Fossil  Fuels)  where  the 
NNE  activities  have  given  the  Union  a  leadership  position  which  national  efforts  would  not  have  done  as 
well.  Particular  examples  of  successful  intervention  are  to  be  found  in  fuel  cell  technology,  wind  turbine 
design  and  combustion  in  automotive  engines.  The  Community  can  play  a  particularly  effective  role  in 

114 

11. 

supporting  long term  and  risky  transnational  research  projects  which  individual  Member  States  might  have 
difficulty  in  supporting. 

Replication  and  commercialisation  have  not  always  been  as  successful  as  would  be  desired,  due  in part  to 
the  low  price  of  fuels  and  to  the  apparent  security  of  supply.  The  response  from  contractors  to  the 
questionnaires,  although  disappointing  in  some  respects,  did  indicate  substantial  outputs  in  the  forms  of 
publications,  theses,  new products,  new processes  and test methods. 

The  scientific  and  technological  base  of  the  Union  has  undoubtedly  been  strengthened  by  the  NNE 
activities  through  the  development  of  research  teams  and  research  facilities,  and  the  promotion  of 
international  co-operation.  In  addition,  the  NNE  activities  are  consistent  with  the  Community's  energy 
policy  and  have  supported  other  Community  policies  on  the  environment,  cohesion,  the  development  of 
the  internal  market  and  sustainable  development.  There  are  in  fact  at  least  twelve  Community  initiatives 
and  instruments  related  to the  NNE  activities  which  serve  to  exploit  and  disseminate  research  results.  The 
linkages  between  these  programmes  and  instruments  are  still  being  developed.  Coordination  between  the 
Community's  activities  and those  of Member  States has not always been  good,  but  is  improving. 

1.6  Conclusions 

The Assessment  Panel's  conclusions  are  summarised  below: 

The  initial  objectives  of the programmes  are  still  valid  against  evolving  scientific,  technical,  industrial 
and  socio-economic  conditions.  These  objectives  will  remain  largely  valid  for  the  Fifth  Framework 
Programme. 
Community-funded  RTD&D  has  contributed  to  almost  all  the  developments  in  the  field  of  NNE  and 
has  also  helped  strengthen  the  Union's  scientifc  and  technical  base.  Funding  has  been  provided  for 
long term  and  risky  research,  which might  not have been  funded  otherwise. 
De facto,  the  two  elements  of  the  Programme  operate  largely  as  two  separate  programmes,  as  they 
originated  in two different  Directorate  General. 
The  management  of  this programme  is considered  by  this  Assessment  Panel  to be  cost  effective.  The 
transparency  of management  is generally  satisfactory. 

-  Both  the  THERMIE  and  JOULE  elements  of  the JOULE-THERMIE  Programme  appear  to be  under 
staffed.  This  has  resulted  in  delays  in  contract  negotiation  and  insufficient  time  for  real  project 
management. 
Delays  in  contract  negotiation  continue  to  be  a  problem  for  contractors  and  appear  to  be  caused  in 
part  by delays  in the contracts  division. 
Clear  quantified  objectives  have  not  been  set  for  the  Programme,  making  if  difficult  to  assess  its 
effectiveness.  It might be difficult  to set such  objectives. 
Co-ordination  of Community  and  national programmes  is not  satisfactory. 
Full use does  not appear  to have been made of all available modes of  action. 
The  project  selection  criteria  and  procedures,  as  well  as  the  objectives  and  scope,  laid  down  in 
Council  directives  have been respected  by the  Commission. 
Literature  provided  for  intending  participants  in the  Programme  has  improved,  but  there  is  room  for 
further  improvement.  There  is  also  scope  for  harmonising  the  procedures  for  project  selection  in 
JOULE  and  in THERMIE. 
It is difficult  to fund  good  projects  which  contain elements  of research  as well  as  demonstration. 
Project  evaluations  do not fully  use metrics  such as milestones  and outputs. 
Evaluation  of  completed  projects  within the JOULE  element  of the  Programme  has  hitherto  been  non 
existent,  but  is now  in train. 

-  The  establishment  of  the  annual  monitoring  exercise  is  of  great  assistance  to  the  programme 

management. 

115 

1.7  Recommendations 

The  Assessment  Panel  recommends: 

11. 

That RTD&D  activities  should continue  in the area  of  NNE. 

The  NNE  Programme  should  be  more  oriented  towards  user  needs,  emphasising  applications  of 
technologies,  but  a  portion  of  the  budget  should  still  be  directed  to  basic  research.  High  quality 
standards  must be maintained  at all times. 

Further  action  should  be  taken  to  harmonise  the  operation  of  the JOULE  and  THERMIE  elements  of 
this  programme,  for  example  by  having  a  unified  management  structure  and  by  having  harmonised 
proposal  evaluation  and  selection procedures. 

Better  integration  of  THERMIE  type  A  and  type  B actions  should  be  assured  to  improve  replication 
rates. 

Monitoring  and  evaluation  procedures  should  be  extended  and  improved  to  include  mid-term  reviews, 
clearly  defined  milestones  and  annual  reporting  with  project  highlights  for  both  the  JOULE  and 
THERMIE  elements.  The  monitoring  panel  should  have  continuity  and  a  member  of  the  monitoring 
panel  should  serve  on the  Assessment  Panel. 

More  use  should  be  made  in the JOULE  element  of  the  Programme  of  all  of  the  modes  of  action  and 
of the  flexibility  open to the programme  management,  such as concerted  actions. 

Adequate  resources  should  be  devoted  to  programme  and  project  management,  inter  alia  to  reduce 
delays  in contract  negotiation. 

Better  co-ordination  with,  and  between,  Member  States'  programmes  should  be  established. 

A procedure  should  be  devised  to deal  with good  proposals  which  overlap the JOULE  and  THERMIE 
elements  of the  Programme. 

116 

11. 

II. 

COMMISSION'S  RESPONSE 

The  Commission  welcomes  this  independent  evaluation  of  Research,  Technological  Development  and 
Demonstration  (RTD&D)  activities  in the field of  Non-Nuclear  Energy  (NNE),  prepared  by  the  high  level 
group  of  experts  chaired  by  Dr.  Eamonn  Kinsella,  and  is  grateful  for  the  effort  shown  by  the  panel  in 
assessing  the  last five  years  of the programme. 

The  report  comments  on an  important  number  of elements  associated  with the  relevance  of the  programme 
objectives,  efficiency  and  effectiveness  of  overall  management  and  implementation  mechanisms.  The 
Commission  views  the  remarks  as  providing  helpful  guidance  and  takes  good  note  of  the  Assessment 
Panel's  conclusions  that  initial  objectives  of the programme  should  remain  largely  valid  for  the  future;  that 
Community-funded  RTD&D  has  contributed  to  almost  all  the  developments  in  the  field  of  NNE  and  has 
also  helped  strengthen  the  Union's  scientific  and  technical  base;  that  project  selection  criteria  and 
procedures,  as  well  as  the  objectives  and  scope,  laid  down  in  Council  decisions  have  been  respected;  and 
that the overall  management  is considered  to be cost  effective. 

The  Commission  welcomes  the  Panel's  recommendation  to orient  the  NNE programme  more  towards  user 
needs,  emphasizing  the  application  of  technologies.  In  fact, 
the  provision  of  high-quality  energy 
technologies  and  services  - which  respond  to the citizens expectations  for  improved  quality  of  life  - and  the 
timely  demonstration  that  these  technologies  and  services  will  limit  environmental  hazards  are  essential 
elements  of the  European  Energy  policy. 

The  Commission  supports  the  Panel's  view  that  the  objectives  of  the  Community's  actions  in  the  domain 
of  NNE  are  served  through  carrying  out  RTD&D  actions  on  rational  use  of  energy,  exploitation  of 
renewable  and  new  indigenous  sources of energy  and also  in the hydrocarbon  sector. 

The  Commission  is  also  aware  that  additional  efforts  are  needed,  at  the  whole  framework  level,  for 
harmonizing  management  and  administrative  procedures  as  well  as  improving  co-ordination  with,  and 
between,  the  various  Member  States'  RTD&D  programmes.  The  Communication  "Inventing  Tomorrow" 
and  the  subsequent  Working  Document  on  the  Structure,  Instruments  and  Objectives  of  the  Fifth 
Framework  Programme  indicate  the  main  lines  of  actions  which  are  intended  to  be  followed  in  this 
respect. 

Finally,  the Commission  views the recommendations as very  constructive,  gearing  both  future  activities 
and  identifying  short  term  adjustments  which  are due to be timely  introduced  in the  remaining  parts  of the 
current  programme. 

117 

11. 

III. 

EVALUATION PANEL 

Dr. Eamonn  KINSELLA 
Mr.  E.  Kinsella  is  Director  of  a  consultance  company  The  Circa  Group  Europe.  He  is  an  expert  in  energy 
and  in  material  technologies.  He  was  a  Research  manager  at  Moy  Insulation  Ltd  and  a  Member  of  the 
National Board for  Science & Technology  (Ireland). 

Mr. Maurice  CLAVERIE 
At  the  CNRS,  Mr.  Claverie  is  responsible  for  the  relations  with  the  Public  Research  Establishments  and 
coordinator  of  the  PIR  (Programmes  Interdisciplinaires  de  Recherche).  He  is  also  advisor  at  the  DGRT 
(Direction Générale de la Recherche et de la Technologie) for the energy and environmental  problems. 

Mr. Dale LAIDLER 
Mr.  D.  Laidler  has  joined  ICI  Corporate  Laboratory  as  Senior  Research  Scientist.  Since  1991,  he  is  the 
External  Collaborations  Manager.  The  role  involves helping  individual  research  groups within ICI  to become 
involved  in academic  and  inter-company  collaborative activities. 

Mr.  Hans Christian  SORENSEN 
Director at the  Energi  og Miljoundersogelse  company,  Mr.  Sorensen  is specialised  in the field of  rational use 
of energy  and use of renewables  in buildings and in the area of combustion. 

Prof. Anastasios  KARABELAS 
Mr.  Karabelas  is  Professor  in  the  Chemical  Enginnering  Department  of  the  Aristotle  University  of 
Thessaloniki.  He is Director  of the Laboratory  of Chemical Process and Plant Design. 

Prof. Dr.  Ing. Dr. Ing. E.h.  Helmut  SCHAEFER 
Mr.  Schaeffer  is Professor  at the  Technical  University  of  Munchen.  He  is responsible  for  the  Research  Unit 
for  Utilities. He has experience  in applied research  in the area of utilisation of energy  in industry,  in domestic 
and transport  (structural  analysis  in energy demand,  energy concepts, ...). 

118 

12. 

1996 FIVE-YEAR ASSESSMENT 

TRANSPORT 

119 

12. 

T. 

EXECUTIVE  SUMMARY 

Recommendations 

1.  It is very  important  that  transport  remains  a distinct  programme,  while  being  strongly  connected  to  other 

areas of the Framework  Programme. 

2.  The  subsequent  overall  transport  RTD  programmes  should  include  transport  related  aspects  like 
technology  development,  energy conservation,  social and environmental  considerations,  land use etc, with 
the various aspects managed by the appropriate DGs within a coherent  framework. 

3.  It  is  important  to  keep  a mixture  of  technology  research  and  policy  research  within  one  overall  transport 

RTD programme. 

4.  Tasks  to be  included  in the  subsequent  programmes  should  be  selected  so that  highest  priority  is given  to 
problems  that  most  effectively  are  handled  on  an  European  level  and  especially  those  which  are  of 
importance  for  the EU transport policy. 

5.  It is important  that the overall programme  has a clear and communicable  vision; there should be  scope  for 
selected  risk taking and also some long range research that spans successive Framework  Programmes. 

6.  The specific  transport programme  should be structured  so that there is a strategic part,  an  "umbrella", 

where problems common to the different  modes, and co-ordination  and competition between modes are 
handled.  Under this  "umbrella"  relations between transport and other related aspects should be  included. 

7.  There  is an urgent  need to increase the number  of staff  to manage the transport  RTD activities;  initially up 
to  the  agreed  staffing  levels  but  a  policy-oriented  programme  of  this  nature  needs  a  higher  ratio  of 
scientific  officers  to  projects  than  in  more  technology-oriented  programmes,  so  agreed  levels  should  be 
raised. 

8.  There  is a need to build  in more flexibility  in the programming  of the budgets.  A certain amount of money 
should  be  kept  aside  within the  specific  programme  to make  it possible  to react  to  changes  in society  and 
policy,  new developments  in technology,  to prolong projects  from  earlier programmes, etc. 

9.  We  strongly  support  the  measures  DGVH  are  proposing  to  take  to  improve  dissemination;  we  emphasise 
the need for  dedicated  in-house staff  with specific  responsibility  to facilitate  dissemination  and  recommend 
that appropriate  resources  are made available. 

10.Concerted  actions  and  task  forces  have  proved  to  be  very  effective  in  enhancing  co-operation  and  co 

ordination,  and these mechanisms should be built appropriately  into future  Framework  Programmes. 

Background 

This  assessment  report  has  been  prepared  according  to  Article  4.2.  of  the  Council  Decision  of  15th 
December  1994  on  the  specific  RTD  programme  in the field of  Transport.  The  programmes  and  initiatives 
covered  by  this  5-year-assessment  are:  EURET,  APAS  and  the  start-up  phase  of  the  much  larger  Transport 
RTD Programme  in FP4. 

The  objectives  of  this  five  year  assessment  are  to  consider  the  relevance,  efficiency  and  effectiveness  of  the 
programmes  and  initiatives  and  the  contribution  of  the  RTD  activities  to meeting  the  Community's  transport 
and other policy  objectives. 

121 

12. 

This assessment  was carried  out between July  and  October  1996. The  Panel examined  project  and  evaluation 
reports  and  the  documentation  associated  with  the  Transport  RTD  Programme.  In  addition,  the  evaluators 
have held  a number  of  meetings  with  project  leaders,  representatives  from  related  EU  projects,  members  of 
the Transport  Research  Committee  (TRC)  and  staff  from  DGVII,  DG XIII  and  Eurocontrol.  The  evaluation 
panel met on four  occasions at the offices  of DGVII, and corresponded  directly at other times. 

The  intention  of  the  transport  research  programme  is to  improve  the  efficiency  of  the  individual  transport 
modes  and  to  find  a  strategy  for  a  faster  integration  and  interoperability.  The  research  covers  both 
passenger  and  freight  transport.  There are other specific  programmes  that are related  to transport,  which  are 
evaluated  separately, but we have included some connections to these other programmes. 

Major  achievements 

It  is  only  from  the  limited  budget  EURET  and  APAS  activities  that  major  achievements  can  be  expected, 
as  the  Transport  RTD  work  is  still  in  its  early  stages.  Examples  of  major  achievements  include  the 
following: 

Intermodal:  design  of  an  innovative  and  efficient  system  of rapid  loading  and unloading  of goods  between 
different  modes  of transport  (SIMET). 
Rail:  development  of  a  harmonised  European  Rail  Traffic  Management  System  (ERTMS)  aimed  at 
overcoming  the  fragmentation  of  rail traffic  management  systems used by the national  railways. 
Waterborne  (maritime):  application  of  advanced  technologies  and  their  integration  with  optimum  crew 
composition,  manning  and  operational  strategies  (ATOMOS). 
Air:  improvement  and evaluation  of air traffic  management  scenarios  (AEGIS). 
Road: use of  cost-benefit  and multi-criteria  analysis for  new  road  constructions. 

In  addition,  the  research  activities  have  been  very  successful  in  encouraging  the  relevant  groups  in 
industry,  transport  operators,  researchers  and  policy  makers  to  work  more  closely  together,  and  in 
fostering  a greater  degree  of co-operation between  organisations  in different  Member  States. 

122 

12. 

II. 

COMMISSION'S  RESPONSE 

The  Commission  welcomes  this  independent  evaluation  of  EU  RTD  in  the  field  of  transport.  The 
recommendations  will  be  taken  into  account  in  orienting  the  further  development  of  transport  related 
research  in the  4th  and  5th  Framework  Programmes  and  will  also  have  an  impact  upon  the  way  in  which 
the results  of the research  and development  are disseminated  and  implemented. 

The Commission  acknowledges  the need  for  a mixture of  "technology"  research  and  "policy"  research.  It 
welcomes  the  panel's  conclusion  that  "the  contribution  of  the  specific  transport  RTD  programmes  to 
assisting  in  the  implementation  of  the  transport  policy  of  the  EU  is  considerable".  This  requirement  to 
support policy  is clearly  set out  in Article  130f  of the  Union treaty  which  refers  to the need  for  research  to 
strengthen  the  scientific  and  technological  bases  of  Community  Industry  and  to  support  other  Community 
policies. 
It  is  also  clearly  acknowledged  in  the  Communication  from  the  Commission  "Inventing 
Tomorrow".1 

The  panel  desires  to  see  a  single  Transport  RTD  Programme  in  FP5.  The  need  for  a  degree  of 
concentration  of  future  research  efforts,  has  been  recognised  by  the  Commission.  The  FP5  proposal  is 
likely  to  see  a  significant  reduction  in  the  number  of  specific  programmes.  This  would  facilitate  the  co 
operation  and  co-ordination  of  research  effort  on  themes  such  as  transport  which  are  currently  situated 
within  four  different  specific  programmes. 

The  Commission  also  notes the  opinion  of the panel  with  regard  to the need  for  an  overall  "umbrella"  for 
transport  RTD  under  which  problems  common  to  the  different  modes,  and  competition  and  co-ordination 
between  modes,  are  clearly  handled.  The  specific  Transport  RTD Programme  has devoted  around  20%  of 
its budget  to Strategic  Research  which  is in part  fulfilling  such a role. 

As  regards  flexibility,  the  Commission  recognises  the  need  for  flexible  programming  and  management  in 
the  context  of  the  Transport  RTD  Programme.  Use  has  already  been  made  of  the  limited  flexibility 
provided  for  by  the  specific  Programme  on  Transport.  Examples  include  the  modification  of  the  funding 
basis  for  a  number  of  tasks  and  the  emphasis  on  aspects  of  research  related  to  road  pricing  introduced  in 
the Third  Call  for  Proposals.  The  creation  of  greater  flexibility,  through  setting  aside  a certain  percentage 
of  funds  within  a specific  programme  to  cover  unforseen  requirements,  will  however  be  considered  when 
preparing  the Fifth  Framework  Programme. 

The  role  of  concerted  actions  and  task  forces  has  rightly  been  recognised  by  the  panel  as  a  means  for 
enhancing  co-operation  and  co-ordination.  The  Commission  intends  to  incorporate  both  of  these  types  of 
actions  in future  Framework  Programmes. 

Finally,  the  Commission  recognises  the  importance  of  dissemination  and  exploitation  in  the  field  of 
transport  RTD.  It  insists,  within the  Transport  RTD  Programme,  that  all projects  include  a work  package 
dedicated  to  the  dissemination  and  exploitation  of  RTD  results. 
In  addition,  a  specific  task  at  a 
programme  level  was  launched  in the  3rd  Call  for  Proposals  on  17 December  1996 which  aims  at a  range 
of measures  to foster  the dissemination  and  exploitation  of all research  results. 

1  COM  (96)  332  final  -  10.07.1996 

123 

IIL 

EVALUATION PANEL 

Professor  Peter Jones 
Director,  Transport  Studies Group,University of Westminster. 

Professor  Bengt  Holmberg 
Professor,  Dept.  of Traffic  Planning  and  Engineering, 
University  of  Lund 

Mr Ambrosius  Baanders 
Deputy  Director, 
Transport  Research  Centre  AW 

Mr Jorge Hernando  Cogollor 
President,  Consultrans  S.A. 

Mr Harri  Kallberg 
Director,  Road  Traffic  Centre. 

Dipl-Ing Konrad  Lukaschek 
Head  of Division,  Signalling  and Traffic  Control  Systems 
SIEMENS  AG Ôsterreich. 

12. 

UK 

SW 

NL 

ES 

SF 

AU 

124 

13. 

1996 FIVE-YEAR ASSESSMENT 

TARGETED SOCIO-ECONOMIC RESEARCH 

125 

13. 

T. 

EXECUTIVE  SUMMARY 

1.1 

The Panel's general findings on the Targeted Socio-economic Research Programme 

Before  the  start  of  the  4th  Framework  Programme,  socio-economic  research  at  Community  level  was 
characterised  by  two  processes:  On  the  one  hand  concrete  social  science  research  and  studies  were 
performed  in  connection  with  the  MONITOR  programme  and  as  an  integral  part  of  some  specific 
programmes.  On the  other  hand,  a broad  discussion  on the potential  importance  of  collaborative  European 
social  science  research  started  in the  late  1980s.  Major  institutional  actors  contributed  substantially  to that 
discourse. 

The  4th  Framework  Programme  followed  the  ambitious  plans  that  were  the  outcome  of  that  intensive 
interactive  process  only  to  a  very  limited  extent.  The  scope  of  the  TSER  programme  is  not  only  much 
narrower  than  the  different  considerations  in  the  beginning  of  the  nineties  have  envisaged,  but  it  is  also 
divergent  from  them  in  so  far  as  it  is highly  fragmented  in  nature.  Thus,  it  appears  more  as  a  first  pilot 
activity  in the area  of socio-economic  research  at European  level  and  less  as a strategic  tool  to  investigate 
into  the  deep  roots  which  give  way  to  exclusion,  poverty,  unemployment,  urban  deterioration  etc.  in 
Europe. 

The  assessment  Panel  has  concentrated  on  the  TSER  programme  as  it  is,  not  on  TSER  as  it  could  or 
should  have  been.  In  addition,  it  must  be  emphasised  that  the  Panel's  assessment  is based  on  insights  and 
impressions  obtained  at  a  very  early  stage  of  the  programme's  development,  when  projects  have  started 
only  a few  months  ago.  Keeping  these points  in mind: 

a.  despite  its  limited  scope,  the  TSER  programme  must  be  considered  as  a  first  important  attempt  to 
improve  the  knowledge  on  a  number  of  important  specific  problems  and  to  encourage  large  scale  co 
operation  in  the  European  social  science  community  thus  providing  inputs  into  the  political  debate  and 
contributing  to the development  of  a European  socio-economic  research  infrastructure; 

b.  although  the  programme  has  seen  many  different  challenges,  mainly  on  the  general  leadership  and 
management  level  in addition to the typical  learning  problems  of  a fairly  new  activity,  a number  of  high 
quality  research  projects  and  thematic  networks  have  been  selected  and  started  to  work  a  few  months 
ago. 

The  TSER  programme  is providing  a new platform  for  collaborative  multi-national,  multi-cultural,  policy-
oriented  and  policy-relevant  research,  which  is a new  feature  for  socio-economic  research  in  Europe.  The 
programme  objectives  are  more  valid  than  ever.  The  Panel  expects  that  relevant  results  will  be  produced, 
in  parts  of  the  programme  already  in  the  very  near  future.  The  Panel  was  pleased  that  in  some  of  the 
projects  some  interesting  results  are  available  already.  The  results  of  the  programme  will  contribute  to  the 
development  of  new  scientific  knowledge  in  the  areas  of  socio-economic  research  covered  by  the 
programme,  and  to  the  improvement  of  Community  policies  in  the  areas  of  research  and  technological 
development,  education  and training,  and  social exclusion  and  integration. 

Since the programme's  specific  objectives  call for  extensive networking  as a special  characteristic  of 
TSER,  a precondition  for  safeguarding  the success will be that the  research/policy  links are  further 
developed  - at the  level of the programme,  the thematic clusters  and the  individual  projects.  First  results 
are promising,  but  appropriate  mechanisms  of  intensive  interaction  and communication  have to be  ensured 
and strengthened  also  in the  future. 

The  TSER  programme  has  been  facing  many  organisational  difficulties  in  its  development  -  already  four 
different  directors  in  its  short  life  time,  changes  in  personnel  and  responsibilities,  and  a  number  of 
different  organisational  structures.  Obviously,  the  Commission  was  not  able  to  ensure  a  harmonious  start 
up phase  of  that  new  activity  until  recently.  For  the  Programme  Committee  the  leadership  problems  at the 
programme  level  in the  first  period  of the programme  development  have been  less then  satisfactory. 

127 

13. 

the  Panel  acknowledges 

that  during  the  implementation  phase  of  the  programme 

However, 
the 
Commission  staff  performed  surprisingly  well  under  very  complex,  turbulent  and  sometimes  confusing 
circumstances,  and  succeeded  to  organise  the  start  up  of  the  programme  -  call  for  proposals,  project 
evaluation  and  selection,  contract  negotiation  - efficiently  and  to  a high  standard.  The  Panel  is satisfied  to 
note  that  for  the  second  Call  for  Proposals  the  recommendations  of  the  1995  External  Monitoring  Report 
have been  followed. 

The  objectives  of  the  TSER  programme  as  defined  in  the  Council  Decision  are  defined  sufficiently  clear. 
In the future,  however,  target  groups  and performance  indicators  should be defined  where  appropriate. 

The present  Panel  is aware  of  the  criticism  by  the previous  1995 External  Monitoring  Report  of  the  TSER 
Programme  that the management  costs  of the TSER  Programme  are too high.  After  one year  of  experience 
the  Panel  takes  a  different  perspective  on  this  issue.  The  specific  nature  of  the  programme  in  connection 
with  its  important  networking  tasks  calls  for  adequate  staffing  -  both  in  numbers  and  in  appropriate 
qualifications.  In  the  very  near  future,  the  situation  will  become  even  worse,  when  the  projects  resulting 
from  the  second  Call  for  Proposals  will be starting. 

The  TSER  programme  has  appropriate  modes  of  action  -  research  projects,  thematic  networks,  concerted 
actions  and  accompanying  measures.  The  Panel  would  welcome  that  thematic  networks  and  concerted 
actions  are  stronger  advertised,  because  these  modes  of  participation  in  the  programme  are  especially 
suited  to  mobilise  and  stimulate  European  co-operation  in  the  socio-economic  research  community  where 
this  was  not  developed  so  far  in  the  past.  In  addition,  the  Panel  strongly  support  the  attempts  to  cluster 
projects  around  themes  following  similar  objectives,  thus  providing  a  mechanism  for  re-integration  of  an 
originally  fragmented  programme. 

The  quality  of  the  selected  projects  leads  the  Panel  to  believe  that  they  will  generate  considerable  added-
value.  Thus,  the  Panel  expects  a  considerable  impact  of  the  programme  -  in  economic  terms  through  its 
future  contribution  to  the  improvement  of  Community  RTD  policies,  and  in  terms  of  inputs  to  the 
development  of  new  European  approaches  in education  and  training  as well  as of  new  social  models.  Also 
its potential  contribution  to harmonisation  of  social  data  has  to be  emphasised.  The  TSER  Programme  has 
initiated  a  process  of  Europeanisation  in  the  different  areas  of  socio-economic  research  addressed  so  far. 
This  can  be  seen  as  an  important  impact  in  itself  already,  because  a  well  developed  European  social 
science  infrastructure  will be needed  in the  future. 

However,  as  has been  said  already,  the  achievement  of  the  objectives  will  to  a large  extent  depend  on  the 
effectiveness  and  efficiency  of the  research/policy  interaction. 

The  Panel  has  identified  deficits  with  regard  to  the  communication,  networking  and  co-ordination  tasks  of 
the  TSER  programme,  in  the  direction  of  other  specific  programmes,  other  Commission  services  and  of 
the  policy  levels  in  the  Community  and  in the  member  states.  These  deficiencies  are  clearly  linked  to  the 
staffing  problem  of the programme. 

The  Panel  strongly  supports  a  dual  approach  to  the  integration  of  social  sciences  in  the  5th  Framework 
Programme: 

.  The  Panel  firmly  recommends  a  separate  specific  socio-economic  research  programme  addressing  a 
well  chosen,  coherent  set  of  themes  that  are  European  by  their  very  nature.  A specific  programme  will 
provide  the  visibility  and  the  integrative  momentum  that  will  be  necessary  to  intensify  the  formation  of 
a  European  social  science  area,  to  support  other  Community  policies  and  to  underpin  the  activities  of 
other  Community  programmes  and  Commission  services. 

.  The Commission  should  continue  and  strengthen the  social  sciences  parts  of  specific  programmes  where 
appropriate.  It  will  be  necessary  to  further  develop  specific  methodologies  for  risk  and  impact 
assessment  in the different  technology  areas,  so as to arrive  at  integrated  planning  systems  that  will help 
avoiding  negative  social,  environmental  and economic  impacts from  the  outset. 

128 

The credibility  of the European  Union as a supra-national  organisation  will depend  to a large extent  on its 
success  in solving  the most challenging  problems  in the different  policy  areas connected  with the  overall 
process of  European  integration.  The Panel  is convinced  that there  is a need  for  integrated,  system-
oriented  approaches  as provided  by a well designed  future  socio-economic  programme. 

13. 

1.2 

The  Panel's  major  recommendations 

In  the  following  the  Panel's  major  policy  relevant  recommendations  are  given  in  a  condensed  form.  A 
much more detailed  list  is given at the end of the  report. 

1.2.1  Contents  of the  TSER  programme 

1.2.1.1 Programme  design,  workprogramme  etc 

1. 

2. 

3. 

4. 

The  Panel  supports  the  notion  of  European  multi-cultural,  multi-disciplinary,  policy-oriented  and 
policy-relevant  socio-economic  research  organised  in  a  separate  specific  programme  for  socio 
economic  research. 
The Panel  recommends  to discuss  in the  Programme  Committee  a revision  of  the  work  programme 
for  the  3rd  Call  for  Proposals,  restricting  the  contents  to  a  few  strategic  themes  and  ensuring 
synergies  between  the different  areas of the programme.  The new  work programme  should  indicate 
well  defined  target  groups,  clearly  defined  objectives,  as  well  as  research  tasks  and  performance 
indicators. 
The  Panel  wishes to draw  attention  to the fact  that cross-European  research  in the  social  sciences  is 
at  a  very  early  stage  of  development.  Therefore,  TSER  thematic  networks  and  concerted  actions 
should  be  particularly  advertised  because  they  are  adequate  means  to  mobilise  the  social  science 
research  community  in Europe. 
The  Panel  identified  an  urgent  need  for  an  overview  of  socio-economic  research  done  in  the 
member  states,  in  the  Commission  and  in  other  specific  programmes.  This  issue  should  be 
discussed  in  the  Ad  Hoc  Advisory  Group  on  Co-ordination  for  the  Targeted  Socio-economic 
Research  Programme. 

1.2.1.2 Future  of  Community  activities in the field  of socio-economic  research 

5. 

6. 

7. 

Since many  of the major  strategic  issues facing  the  EU  over  the coming  years  have  socio-economic 
aspects,  and  since  this  area  of  the  social  sciences  is  so  far  underdeveloped  in  Europe,  the  Panel 
stresses  the  crucial  importance  and  pioneering  role  of  a  Community  socio-economic  research 
programme.  The  Panel  strongly  supports  the  idea  of  a  dual  approach:  to  have  a  separate  well 
designed  and  coherent  specific  socio-economic  research  programme  in  the  5th  Framework 
Programme  underpinning  research  and  study  activities  by  other  specific  programmes  and 
Commission  services,  and  to  further  develop  socio-economic  research  as  parts  of  specific 
programmes  where  appropriate. 
To  develop  solutions  for  the  most  pressing  European  challenges  calls  for  system-oriented 
approaches  building  on  multi-dimensional  perspectives,  where  the  social  dimension  is  essential. 
These  issues  are  particularly  important  in  the  light  of  the  tensions  that  are  building  up  in  the 
connection  of deepening  and enlargement  of the European  Union,  which  have the potential  to place 
barriers  to  further  realisation  of  the  goals  of  the  Maastricht  Treaty.  The  social  sciences  have  to 
make  important  contributions  both  from  the  point  of  view  of  their  broad  range  of  special 
disciplinary  aspects  and  their  ability  to  develop  appropriate  social  processes  and  instruments  for 
shaping  the  future.  Since  the  problems  are  truly  European  in  nature  they  have  to  be  dealt  with 
collaboratively  at European  level. 
In  preparing  the  future  programme  not  only  the  definition  of  contents  is  essential  but  also  the 
design  of  an  appropriate,  well  balanced  set  of  instruments  and  modalities  for  participation: 
Thematic  networks,  concerted  actions,  research  projects,  observatories  and  accompanying 
measures,  especially  to ensure continuous  communication  and  interaction  with target  audiences. 

129 

8. 

9. 

10. 

The  preparation  of  the  future  socio-economic  research  programme  has  to  be  realised  in  close  co 
operation  with the scientific  community  and potential users of  results. 
In  the  preparation  of  FP5  the  Commission  should  follow  an  integrative  approach,  ensuring  the 
active  support  and  commitment  of  all  Commission  services  with  a  specific  interest  in  socio 
economic  research. 
to  integrate  into  the  research  activities 
The  Panel  recommends  that  appropriate  means  are  offered 
groups  from  those  countries  from  central  and  eastern  Europe  who  are  applying  or  have  already 
applied  for  EU  membership. 

13. 

1.2.2  Management  and  procedures 

11. 

12. 

13. 

14. 

15. 

16. 

17. 

18. 

19. 
20. 

21. 

To  better  cope  with  the  intellectual  aspects of  research  management  at the  research/policy  interface 
going  beyond  mere  administrative  monitoring  on  the  level  of  project,  project  clusters  and 
programme,  more  staff  with  practical  experience  in  social  science  research,  and  with  capabilities 
and  skills  in moderating  interactive  processes  will be  necessary. 
As a consequence  of  the  problematic  personnel  situation  of  the TSER  programme,  the  Panel  points 
at  an  urgent  need  to  clarify  the  staffing  situation  of  the  TSER  programme  and  to  take  respective 
steps  to  ensure  that  the  programme  is  equipped  with  adequate  personnel  resources  for  its  core 
activities  in  accordance  with  the  specific  objectives  and  tasks  of  TSER  and  within  the  limits  of  the 
indicative  budget  breakdown  defined  by  the  Council  Decision.  A  personnel  recruitment  and 
development  strategy  should  be  prepared  in the very  near  future. 
The  Panel  recommends  that  procedures  followed  in  the  evaluation  and  the  final  selection  of 
proposals  are  clearly  described  in  the  Information  Package.  The  process  should  be  organised  in a 
more  transparent  way,  involving  independent  experts  to  support  and  advise  the  Commission 
services  and  the Programme  Committee  in the preparation  of the  short  list. 
The  Panel  recommends  that  the  efforts  of the  Commission  services  towards  thematic  clustering  of 
projects  concerning  the  same  strategic  issues  across  different  sub-areas  and  between  areas  should 
be  further  developed.  They  will  support  inter-project  communication,  integration  and  co-ordination 
and  ensure  the  future  impact  of  the  programme  both  at  the  policy  level  and  within  the  scientific 
community. 
The  Panel  particularly  welcomes  that  Project  Progress  Seminars  are  convened  by  the  Commission 
services  regularly,  approximately  every  sixth  month.  Such  seminars  are  appropriate  means  to 
the  policy  makers,  other  Commission  services  and  Community 
support  networking  with 
programmes  and  initiatives,  and  the  scientific  community.  In  addition,  also  individual  projects 
should  be  encouraged  to  ensure  interaction  with  possible  users  of  their  results  -  especially  in  the 
in  their  projects  or  to  develop  other  adequate  means  for 
integrating  them 
policy  area: 
communication,  interaction  and  networking. 
The  Panel  feels  that  the  TSER  programme  should  contribute  to  the  development  of  Community 
RTD  strategies  in  a  well  organised  and  co-ordinated  way.  Communication  with  CERT,  CREST, 
IRDAC  and  ESTA  are  of  particular  importance.  In  addition,  the  co-operation  with  the  Cellule  de 
Prospective  and the IPTS should be strengthened  and further  developed. 
There  is an urgent  need  for  taking  a decisive  step to activate  ETAN.  Adequate  measures  have to be 
taken  to  tap  not  only  into  the  results  of  the  TSER  projects  but  also  into  socio-economic  research 
activities  of  other  specific  programmes  and  Commission  services,  the  IPTS,  and  other  actors  in the 
field  both  at the  level of the Community  and  in the member  states. 
The  Panel  feels  that  exchange  of  information  and  co-operation  between  the  TSER  programme  and 
socio-economic  research  activities  in other  specific  programmes  and  by  other  Commission  services 
should  be  strengthened.  The  TSER  programme  should  have  a  facilitating  and  moderating  role  in 
that  context. 
Co-ordination  and  collaboration  with COST should be further  developed. 
The  integration  of  young  promising  researchers  and  research  teams  in  projects  with  well-
established  institutions  is welcomed  and  should be further  supported. 
The  Panel  is concerned  that  a very  specific  issue  arises  in  relation  to the  derivation,  harmonisation 
and  dissemination  of databases  in the  socio-economic  area.  In the  mainstream  areas  of  economics, 
the  national  statistical  agencies  are  the  main  bodies  functioning  in  this  area,  and  are  very  well 

130 

13. 

resourced.  However,  in  areas  II  and  III  of  the  TSER  programme  there  is  a  need  to  generate  new 
quantitative  and  qualitative  data  and  to  harmonise  them  across  a  range  of  member  states  in  a 
context  where  national  agencies  are  less active.  Thematic  networks  and  concerted  actions  should  be 
advertised  to  support  that  problem  area.  Cheap  or  free  access  to  EUROSTAT  data  should  be 
provided  for  funded  projects,  networks  and actions. 
The  social  sciences  being  organised  largely  in  very  small  units  efficient  services  for  better 
information,  advice  for  the  preparation  of  proposals  and  support  for  partner  search  should  be 
further  developed.  In  the  member  states  the  network  of  National  Contact  Points  should  be  further 
strengthened  and  activated  where  necessary.  In addition,  co-operation  with  other  existing  networks 
should  be  developed. 
The Panel proposes  that  at the end  of that  first  assessment  round  a feedback  process  is organised  to 
draw  lessons  from  the experiences  the different  panels made. 

22. 

23. 

131 

13. 

II. 

COMMISSION'S  RESPONSE 

The  Commission  welcomes  the  Panel's  report  as  an  essential  contribution  towards  the  assessment  of 
Community  research  activities  in  the  field  of  socio-economic  research,  and  for  providing  detailed 
recommendations  regarding  the  future  development  of  such  activities  within  the  4th  and  5th  Framework 
Programmes. 

In particular,  whilst  wishing  to  recall  to mind  the  highly  dynamic  nature  of  policy  relevant  research,  the 
Commission  notes  the  opinion  of  the  Panel  that  the  rationale  underpinning  a  specific  programme  in  the 
field  of  targeted  socio-economic  research  is  fundamentally  sound  and  that  the  Programme's  objectives 
remain  not just  valid  but  also highly  pertinent. 

In  this  context,  it  is  noted  with  satisfaction  that  in  the  opinion  of  the  Panel  a  number  of  high  quality 
research  projects  and  thematic  networks,  with  the  potential  of  generating  considerable  added  value,  have 
been  selected  following  the  first  call  for  proposals  launched  under  the  TSER  Programme;  and,  that  the 
Panel  considers  that  a  number  of  these  activities  are  already  producing  interesting  and  policy  relevant 
results. 

As  regards  the  preparation  for  the  forthcoming  third  call  for  proposals  for  research  actions  the  Panel 
recommends  that  the  work  programme  of  the  TSER  Programme  should  be  revised  in  order  to  restrict 
coverage  to  a  number  of  key  strategic  themes.  The  Commission  also  foresees  the  need  to  focus  on  key 
issues  and  will  consider  the  recommendation  to  revise  this  work  programme  whilst  respecting  also  the 
need  to  take  into  consideration  the  outcomes  of  the  first  and,  the  recently  launched,  second  calls  for 
proposals. 

Moreover,  and  in  the  context  of  the  medium-term  development  of  socio-economic  research  carried  out  at 
the  European  level,  the  Commission  welcomes  the  Panel's  recommendation  that  social  sciences  should  be 
developed  within  the  5th  Framework  Programme  through  a  well  defined  social  sciences  research 
programme  addressing  a  coherent  set  of  social  challenges  facing  Europe  from  a  policy  oriented 
perspective,  as  well  as  being  developed  as  an 
integrated  part  of  other  specific  programmes  where 
appropriate. 

Finally,  with  respect  to  the  Panel's  other  recommendations  and  in  particular  those  concerning  the 
operational  and  management  aspects  of the  Programme,  the  Commission  wishes  to  commend  the  Panel  on 
its  objectivity  and  frankness.  The  Commission  undertakes  to  address  these  issues  and  its  services  will 
report  on the measures  to be taken  as part  of the annual monitoring  exercise. 

132 

13. 

III. 

EVALUATION  PANEL 

Manfred  Horvat  (Chairman) 
1993  -  :  Director,  BIT  (Bureau  for  International  Research  and  Technology  Co-operation);  Member  of 
INNOVATION  and  TMR  Programme  Committees;  Member  of  ACTS  Monitoring  Panel  1995  and  1996; 
1995  -  :  Vice-President,  SEFI  (European  Society  for  Engineering  Education);  1993-94:  Visiting  Fellow, 
University  of  Oxford;  1985  -  : university  lecturer  in  Technology  Assessment  and  Environmental  Impact 
Assessment  (Vienna  University  of  Technology,  University  of  Linz);  1980  -  :  Director  (at  present  on 
leave),  University  Extension  Centre,  Vienna  University  of  Technology  (University-industry  technology 
international 
transfer, 
programmes  and projects  in research  and  education) 

technology  assessment,  continuing  engineering  education,  management  of 

Prof Francisco Alvira-Martin 
Professor  of  Sociology,  head  of  Department  of  methodology  and  research  techniques,  Universidad 
Complutense,  Madrid;  he  specialised  in  programme  evaluation  as  a  visiting  scholar  in  North-Western 
University  with  Prof  R.B.  Boruch  and  has written  two  books  and  more  than  five  articles  on  the  evaluation 
research  methodology;  actually  engaged  in  the  evaluation  of  the  Drug  Abuse  Plan  of  Madrid;  1995: 
independent  expert,  panel  that  wrote  the draft  for  the  Evaluation  Indicators  System  of  the  Spanish  National 
Science  Plan. 

Dr John Bradley 
Economist  and  Research  Professor,  Economic  and  Social  Research  Institute,  Dublin;  IRL;  specialised  in 
labour  market  economics  and the economic processes  of cohesion  within the  EU  periphery 

Prof Pierre Desmarez 
Université  libre  de  Bruxelles;  Doctor  in  social  sciences;  Director,  Centre  de  sociologie  et  d'économie 
régionales;  research  topics:  work  and  (un)employment,  work  histories,  transitions  from  school  to work 

Dr Luigi Felici 
1986  -  :  Director,  ANCE,  IT  (strategic  programmes  in  the  field  of  urban  renewal  market,  economic 
analysis  of  infrastructure  investments,  R&D  programmes  in construction  industries);  before  1986  L.  Felici 
had  a  leading  role  in  Confidustria's  industrial  R&D  programmes,  Mezzogiorno  industrial  development 
strategies,  environmental  problems 
industrial  policy  options,  external  affairs  and 
communication);  1979  - 94:  member  of  FAST/MONITOR  Management  Committee,  nominated  by  Italian 
Research  Ministry 

industry, 

in 

133 

14. 

1996 FIVE-YEAR  ASSESSMENT 

COOPERATION WITH THIRD COUNTRIES 

135 

I. 

EXECUTIVE SUMMARY 

A.  Brief  history  of the INCO programme  and scope of the Five-Year  Assessment 

14. 

The  Second  Activity  of the  Fourth  Framework  Programme  (FP4)  is implemented  by  a specific  programme 
of  research  and  technological  development  including  demonstration  (RTD)  with  third  countries  and 
international  organizations.  This  specific  programme,  adopted  by  the  Council  of  Ministers  on  23 
November  1994 and  known  as the  INCO  (International  Cooperation)  programme,  brought  together  for  the 
first  time  all  EU  RTD  cooperation  with third  countries  with the  exception  of  the  nuclear  field.  The  INCO 
programme  is  complemented  by  access  to  the  specific  programmes  of  the  First  Activity  for  all  European 
and  some  non-European  participants  in accordance  with  general  rules  established  by  the  Community.  The 
cooperation  now  contained  within  the  INCO  programme  was  previously  carried  out  through  a  number  of 
different  programmes.  In  developing  countries  three  different  instruments  of  scientific  and  technological 
(S&T)  cooperation  were  replaced  by the single programme  INCO-DC.  In  1991 the  Community  embarked 
on  a  major  expansion  in  RTD  collaboration  with  the  countries  of  Central  and  Eastern  Europe  (CEE)  to 
support  the  integration  process  and  in  1992  launched  the  PECO/COPERNICUS  programmes  which  were 
later  extended  to  the  New  Independent  States  (NIS)  of  the  former  Soviet  Union.  Collaboration  with  non-
European  industrialized  countries  has  developed  in  a  variety  of  ways  reflecting  mutual  interest  within  or 
outside  S&T cooperation  agreements.  INCO  is a horizontal  programme  which  cuts  across  technical  fields 
and 
scientific 
organizations. 

linkages  with  COST,  EUREKA  and 

the  Community's 

international 

incorporates 

This  five-year  assessment  covers  the  activities  and  management  of  the  INCO  programme  and  its 
predecessors  during  the  period  1991-early  1996  and  incorporates  the  final  evaluation  under  the  Third 
Framework  Programme.  "The  four  main  geographical  sectors  covered  by  the  INCO  programme  and 
considered  in the  report  are: 

Area  Ah  Cooperation with other Fora for  European  S&T Cooperation: 

COST, EUREKA,  International  S&T Organizations and Institutions 

Area  A2:  Cooperation with the  Countries of Central and Eastern Europe 

and  with the New Independent  States of the former  Soviet Union 

Area  B:  Cooperation with non-European Industrialized  Third Countries 
Area  C:  S&T  Cooperation with the Developing Countries 

The  assessment  panel  has  examined  in  detail  the  results  and  progress  made  in  each  main  sector  and 
considered  what  useful  lessons  might  be  drawn  from  the  Programme  as  a  whole.  The  full  report  of  the 
Panel  is available  in English  as a separate  document. 

B.  The panel's comments  and  findings 

1.  Increasing  globalisation  of the  economy  and the  spread  of the  Information  Society  are  ensuring  that,  at 
the threshold  of the 21st century,  international  cooperation  in science  and technology  is more  intensive  and 
widespread  than  ever  before.  A  high  level  of  international  S&T  activity  is  required  to  strengthen  the 
science  and  technology  basis  of  EU  industry  and  to  promote  all  the  research  actions  necessary  to  fulfil 
Community  policies.  The  Panel  considers  that  bringing  together  the  various  programmes  of  international 
collaboration  within  INCO  has  resulted  most  importantly  in  greater  coherence  and  visibility  for  EU  S&T 
international  cooperation. 

2.  The  Panel  finds  that  the  programmes  implemented  in  the  period  under  review  were  generally  of 
satisfactory  relevance  to  the  objectives  laid  down.  A  substantial  number  of  concrete  results,  most  with 
potential  for  further 
research,  have  been  obtained  and  disseminated.  Some  significant  scientific 
achievements  have  been  recorded  (Annex  11).  A  proper  assessment  of  industrial  exploitation  would 
require  a longer  time  scale but  it is not  likely  to be  large.  The burdens  of  management  were  increased  by 
oversubscription,  communication  difficulties,  local  conditions  in some partner  countries  and  the  overriding 

137 

14. 

requirements  of  accountability  and  quality.  The  formal  procedures  followed  by  the  Commission  were  not 
conducive  to efficiency  improvements  in some areas. 

3.  The  Panel  was  impressed  by  the  results  that  have  been  gained  by  COST  and  supports  the  existing 
relationship  between  COST  and  INCO.  The  joint  initiative  in  vaccine  research,  for  example,  has 
In  the  recent  past 
facilitated  long-term  cooperation  between  the  scientific  research  sector  and  industry. 
there  have  been  notable  initiatives  in  the  chemistry  domain.  COST  enables  R&D  collaboration  between 
scientific  research  bodies  and  industry  to  take  place  in  a  flexible  manner  as  recommended  in  the 
Commission  document  Coordination  through  Cooperation  .  COST also provides  a nursery  for  projects  in 
specific  programmes,  is  a  useful  vehicle  for  participants  from  outside  the  EU,  and  has  helped  to  shape 
elements  such  as ESPRIT  and  FLAIR  in the Community's  Framework  Programme. 

4.  Because  of  radically  different  selection  procedures  and  time-scales  collaboration  between  EUREKA 
and  the  Community's  programmes  requires  exceptional  efforts  to  achieve  results  but  might  be  improved 
through  procedural  modifications  (see  Recommendation  2 in Part  C of this  Executive  Summary). 

5.  The  Panel  considers  that  both  the  COPERNICUS  and  INTAS  programmes  were  essential,  temporary 
responses  to urgent  needs  arising  in  Central  and  Eastern  Europe  (CEE)  and  in the  New  Independent  States 
(NIS).  These  programmes  have  helped  the  ongoing  process  of  stabilising  research  potential  in  CEE 
countries  and  in  the  NIS  and  have  introduced  many  CEE/NIS  scientists  to  modern  methods  of  scientific 
research  in  a  competitive  environment.  The  economic  impact  of  these  programmes  in  many  CEE/NIS 
countries  has  been  very  limited  because  of  local  deficiencies  in  infrastructure  and  lack  of  suitable 
mechanisms  for  exploitation.  High  priority  should  now  be  given  to  increasing  PHARE  and  TAOS 
support  for  essential  structural  reforms  in  RTD  and  in the  social  and  industrial  applications  of  RTD.  This 
major  challenge  requires  determined  coordinated  action  and  a  constructive  partnership  with  the  CEE/NIS 
authorities  before  INCO  objectives  in the economic,  social  and  environmental  sectors  can be attained.  The 
potential  benefits  for  the  whole  of  Europe  are  large. 

6.  Progress  towards  participation  on  an  equal  footing  in  the  First  Activity  by  CEE  research  teams 
comparable  in standing  with  western  groups  should  be  encouraged  by  the  EU  whenever  this  has  a  realistic 
basis.  A  separate  adhesion  fund  outside  the  Framework  programmes  will  be  required  to  support  this 
participation.  The  transition  to  fully  effective  research  partnership  can  be  accelerated  by  phasing  out  the 
COPERNICUS  programme  by the end  of  FP4. 

7.  The  NIS,  which  retain  a  wealth  of  scientific  expertise,  particularly  in  Russia  and  Ukraine,  require  a 
research 
different  approach.  The  useful  experience  built  up  by  INTAS  in  establishing  co-funded 
programmes  provides  a basis  for  continuing  collaboration  with  important  research  centres,  including  case-
by-case  participation  in specific  programmes. 

8.  Despite  intensive  efforts  by the  Commission  to publicise  EU  collaboration  programmes  within  the  CEE 
countries  and  the  NIS there  is still  a very  low  level  of CEE/NIS  participation  in other  specific  programmes 
and  widespread  lack  of  understanding  among  individual  scientists  of  the  procedures  and  opportunities 
available.  During  the  transition  period  these  efforts  need  to  be  strongly  reinforced  through  all  available 
information  channels  by  very  clear  and  unambiguous  statements  of  eligibility  for  participation  in  different 
programmes  with  practical  details  of costing  and payments  procedures. 

9.  The  Panel  considers  that  collaboration  with  non-European  industrial  countries  and  emerging  economies 
It  supports  the  views  expressed  in  the  1995  IRDAC  report  that  all 
is  rapidly  growing  in  importance. 
Community  R&D  programmes  should  be  open  to  participation  by  organizations  from  outside  Europe  on a 
case-by-case  basis  provided  that  European  industry  in  general,  and  competitiveness  in  particular,  can 
benefit  and  that  reciprocity  and  suitable  IPR  arrangements  can  be  established.  Emerging  economies, 
which  are  currently  eligible  to participate  in the  developing  countries  programme,  should  also  be  included 
under  the  same  conditions  in  respect  of  fields  where  they  have  achieved  high  S&T  standards  and 
performance.  The  Panel  agrees  with  the  Commission's  current  policy  of  negotiating  S&T  cooperation 

38 

14. 

agreements  with  foreign  governments  whenever,  on  the  basis  of  wide  consultation  with  representatives  of 
science  and  industry,  there  is strong  evidence of mutual  interest between  the prospective  partners. 

10.  The  original  aims  of  the  Japan  S&T  Fellowship  Programme  have  largely  been  accomplished  and, 
taking  account  of  existing  bilateral  schemes,  the Panel  considers  that  this Programme  should  henceforth  be 
focused  on  co-funded  and  Japanese-funded  arrangements  for  carefully  selected  candidates  and  host 
research  laboratories  (university,  national  and  industrial)  of the highest  quality. 

11.  The  Panel  has been  impressed  by  the  high  reputation  of  the  STD/INCO-DC  programme  among  both 
coordinators  and  participants  deriving  from  the  integrity  of  the  evaluation  process,  the  creation  of 
successful  partnerships  and  the  applicability  of  the  research  projects.  The  current  practice  of  linking 
postgraduate  training  (leading  to  a  higher  degree)  to  INCO-DC  research  projects  should  continue.  The 
Panel  considers  that  the  quality  of peer  reviewing  is a most  valuable  asset  of  INCO-DC  and  that  it  is in the 
strong  interest of  Europe  to be seen to practise  even-handedness. 

12.  While  INCO's  evaluation  procedures  are  generally  well  regarded  they  are  expensive  in  terms  of 
resources.  The  contribution  of  information  and  communications  technologies  to  the  submission  and 
evaluation  of proposals  has yet to be proven  but the  Panel  considers  that the potential  benefits  are  large  for 
all INCO-administered  programmes. 

13.  The  Panel  considers  that  globalisation  is  often  driven  by  short-term  economic  factors  and  by  the 
prospect  of  access  to  cheap  labour  in  developing  countries.  INCO  has  an  important  role  in  ensuring  that 
basic  and  applied  sciences  are  not  forgotten,  particularly  in  matters  that  relate  closely  to  regional  issues 
and problems. 

14.  In  the  opinion  of  the  Panel  the  low  level  of  direct  industrial  participation  in  the  INCO  programme 
arises  from  a combination  of thematic  and  structural  causes.  The  IRDAC  report  of  October  1996  contains 
recommended  actions  which  should  lead  to  a  greater  involvement  of  industry  in  the  cooperation 
programme  and  to the creation  of  links that  are  favourable  to  innovation. 

15.  The  breadth  and  complexity  of  the  INCO  programme  distinguish  it  from  most  other  specific 
programmes  and  impose  considerable  management  demands  on  Commission  staff.  Constant  and  close 
collaboration  is necessary  between  DG  XII-B and  professional  staff  in  other  directorates  of  DG  XII  and  in 
other  DGs  dealing  with  individual  programme  elements.  In  the  view  of  the  Panel  the  importance  of  the 
INCO  programme  requires  a  strengthening  of  the  role  of  the  INCO  Programme  Committee  and  a 
considerably  higher  publicity  profile. 

16.  In  considering  coherence  between  EU  and  national  S&T policies  the  Panel  has  come  to  the  view  that 
INCO  has  a  particular  contribution  to  make  in  problems  of  a  truly  regional  dimension  which  are  not 
normally  tackled  by bilateral  programmes  and  for  which  scientific  networking  is of paramount  importance. 
INCO  programmes  can  usefully  complement  national  programmes  by  setting  up  European  research 
networks  on  priority  topics  and  through  interdisciplinary  regional  projects  containing  both  theoretical  and 
applied  research  and  requiring  a  balanced  EU/  third  country  input.  Coherence  requires  adequate 
knowledge  on both  sides and  an effective  two-way  flow  of  information.  The  Panel  supports  the  INCOPOL 
study  which  aims to collect  and  synthesise  information  on national  activities  in external  R&D  cooperation. 

C.  The panel's recommendation 

COST.  EUREKA.  International  S&T  organizations 
1.  In  view  of  the  potential  of  COST  for  future  expansion,  its  high  relevance  to  EU  policy  and  beneficial 
interaction  with  the  Framework  Programme,  an  increase  in  resources  for  the  COST  Secretariat  should  be 
considered  and  measures  taken  to  simplify  the  organizational  and  administrative  procedures  as  far  as 
possible. 

139 

14. 

2.  The  scope  for  expanding  cooperation  between  EUREKA  and  Community  projects  should  be  explored. 
For  this  purpose  a pilot  scheme  providing  fast-track  procedure  for  Community  participation  in  support  of 
the  research  part  of  labelled  EUREKA  projects  should  be  established  with  limited  fund  allocation  and 
duration.  The outcome  of this scheme  should be used  as an input to determine  the  extent  of  further  support 
aimed  at strengthening  collaboration between the Community  and  EUREKA. 

3.  Without  evidence  of  clear  added  value  the  Panel  considers  that  the  limited  activity  between  INCO  and 
international  S&T  organizations  does  not  at  present  justify  a  separate  category  within  the  INCO 
programme. 

CEE countries  and the  NIS 
4.  Moves  in  CEE  countries  and  the  NIS  to  reform  the  S&T  sector  need  to be  accelerated  and  reinforced, 
through  close  collaboration  between  INCO,  PHARE/TACIS  and  the  countries  concerned,  in  three  main 
directions: 
• 

restructuring  actions  aimed  at  concentrating  S&T  at  a  small  number  of  centres  of  excellence  with 
advanced  equipment  and  high-calibre  scientists; 

•  assisting  in  the  reorganization  of  S&T  administration,  giving  emphasis  to  modern  transparent  methods 
of  science  management,  competitive  grants  and  peer  review,  and  complementing  what  is already  being 
done  bilaterally; 

•  encouraging  the  application  of  S&T  in  society  and  the  economy  through joint  efforts  in  tackling  socio 
economic  problems.  Particular  attention  needs  to be  given  to  IPR  protection  as  a means  of  stimulating 
innovation. 

5.  The  COPERNICUS  programme  in  CEE  countries  and  in  the  NIS  should  be  phased  out  by  the  end  of 
FP4.  During  the  remainder  of  the  COPERNICUS  programme  much  stronger  focusing  of  calls  on 
priorities  not  covered  by  other  specific  programmes,  particularly  of  a  regional  nature  and  established  in 
negotiation  with  R&D  funding  agencies,  should  be  put  into  effect  together  with  the  transfer  in  appropriate 
cases  of  the  scientific  coordinator's  role  to  a CEE/NIS  scientist.  Following  the  end  of  the  COPERNICUS 
programme  funds  should  be  made  available  for  more  generous  support  for  networks,  conference 
attendance,  modernisation  of  equipment  at  selected  centres,  and  other  measures  so  that  the  amount  of 
contact  remains  at the high  level  sought by both  eastern  and western  scientists. 

6.  The  function  and  objectives  of  INTAS  should  be  urgently  reviewed  so  that  it  is  able  to  undertake  a 
revised  role  in  FP5  in  accordance  with  the  need  to  help  restructure  the  S&T  sector  in  the  NIS,  in 
collaboration  with  TACIS  and  the  countries  concerned,  intensify  high-quality  collaboration  with  NIS 
scientists,  continue  co-funding  arrangements  and  promote  NIS  participation  in  the  First  Activity.  A joint 
analysis  with  the  Russian  authorities  of  important  concentrations  of  civilian  research  outside  Moscow  and 
St.  Petersburg  would  help  to  increase  the  potential  for  mutually  beneficial  collaboration. 
INCO  should 
also  identify,  together  with  TACIS,  reform-minded  funding  agencies  in  the  other  NIS  countries  with  a 
view  to  setting  up  collaborative  links  along  the  lines  of  the  relationship  between  INTAS  and  the  RFBR 
(Russian  Foundation  for  Basic  Research). 

Developing  countries 
7.  The  original  level  of  funding  S&T collaboration  with developing  countries  (DCs)  should  be  restored  in 
view  of the high  relevance  of the  results  of  INCO-DC  to current  needs,  the  growing  importance  of  S&T to 
solving  regional  problems  (with  world-wide  implications),  and  the  need  to  fund  a  substantially  greater 
number  of A-rated  proposals  and  sustain  the results  already  achieved. 

8.  The  basic  objectives  of  INCO-DC  and  the  major  development  areas  of  health,  agriculture  and 
environment  should  be  maintained.  To  reduce  oversubscription  these  areas  should  be  more  strongly 
technology  issues,  which  benefits  not  only  industrial  development  but  adds 
focused.  The  fourth  area 
value to all the cooperation  programmes,  should be given higher priority  so that the benefits  resulting  from 
combining 
technologies  and  from  Information  Society  trends,  in  particular,  can  be  made  more  widely 
accessible  in developing  countries. 

140 

14-

9.  High  priority  should  also  be  given  to  INCO  collaboration  with  EU  programmes  that  support 
infrastructure  developments,  such  as  ALA,  MEDA  and  LOME,  in  order  to  raise  awareness  of  this  issue, 
create  stronger  links  between  infrastructure  investment  and  supporting  RTD  projects,  and  encourage 
appropriate  actions  in developed  countries. 

10.  In  the  light  of  requests  from  the  developing  countries  the  role  of  DC  participants  in  setting  priorities, 
formulating  and  evaluating  proposals,  and  in the  scientific  management  of projects  should  be  strengthened 
and the possibility  of entering  into co-funded  arrangements  should  be  examined. 

Evaluation 
11. 
In  order  to  make  the  evaluation  procedures  more  transparent  and  help  newcomers  to  Community 
research  programmes  learn  through  participation  in the  application  process,  prompt  notification  should  be 
given to proposers  of unsuccessful  projects  together  with  reasonable  feed-back  on their  rating. 

12.  A  pilot  trial  on  the  use  of  information  and  communications  technologies  in  the  submission  and 
evaluation  of  proposals  should  be  carried  out  in  a  future  call  to  determine  whether  the  quality  of  the 
preliminary  stages of the evaluation  process,  including  the necessary  confidentiality,  can be preserved  with 
a view  to preparing  the ground  for  shorter,  more  focused  meetings  in Brussels. 

EU and  national  S&T policies 
13.  Continuous  monitoring  of  RTD  actions  on  a  bilateral  basis  between  Member  States  and  countries 
participating  in  PECO/COPERNICUS  and  INCO-DC  programmes  is  desirable  in  order  to  adhere  to 
subsidiarity  and  avoid unnecessary  duplication. 

INCO  Programme  Committee 
14.  The  mandate  of  the  INCO  Programme  Committee  should  be  carefully  examined  and  revised  where 
necessary  in order  to strengthen  the  Committee's  role  in the development  of  a visible  and  effective  INCO 
programme  and  to  allow  time  for  the  Committee  to  consider  matters  of  a  strategic  and  long-term  nature. 
The  Commission,  with  the  help  of  members  of  the  INCO  Programme  Committee  in  their  own  countries, 
should  make  strenuous  efforts  to  raise  the  profile  of  INCO  by  widely  publicising  its  aims  and  objectives 
and  the opportunities  it provides  for  scientists  to engage  in international  collaboration  on  relevant  issues. 

Administration 
15.  The  Panel  strongly  recommends  that the Commission  give urgent  consideration  to the  introduction  of a 
comprehensive  Management  Information  System  for  the  INCO  programme  capable  of  giving  at  any 
moment  a bird's-eye  view  of  main  trends  and  indicators  and  the  status  of  projects.  This  management  and 
monitoring  tool  would  facilitate  coherence  among  all  DGs  actively  concerned  with  the  programme,  and 
enable  the  INCO  Programme  Committee  and  future  monitoring  and  assessment  panels  to  focus  rapidly  on 
critical  issues.  Since  some  areas  of  the  cooperation  programmes  appear  to  lack  effective  monitoring  and 
reviewing  procedures  for  ongoing  projects  a system  of mid-term  monitoring  on a sampling  basis,  including 
independent  financial  auditing,  should be  implemented  forthwith  and  publicised. 

16.  In  order  to  streamline  the  administration  processes  within  the  Commission,  a  subject  of  continuing 
concern  to  both  coordinators  and  participants,  approval  for  projects  within  a  defined  ceiling  should  be 
given  fast-track  treatment.  Consideration  might  be  given  to  modifying  the  existing  delegation  and 
signature  procedures  within  the  Commission  consistent  with  the  need  to  maintain  transparency.  The 
Community  cannot  afford  to  show  inefficiency  in  its  own  operation  and  at  the  same  time  urge  reforms  on 
its DC and  CEE/NIS  partners. 

Assessment  and  monitoring 
17.  The  Panel  considers  that  in  the  five-year  period  under  review  the  INCO  programme  and  its 
predecessors  were  necessary  to  meet  the  circumstances  of  the  time  and  have  substantial  achievements  to 
their  credit.  In  a  rapidly  changing  world  there  will  be  a continuing  need  for  adjustments  within  the  four-
year  period  of  a  Programme.  In  regard  to  the  CEE  countries  and  the  NIS,  the  emerging  economies  and 
the  developing  countries  there  will  be  many  new  challenges  in the  coming  few  years.  In  this  situation  the 

141 

Panel  recommends  that both the monitoring  and assessment exercises on the lines put forward  by CREST 
should  remain essential components of the INCO programme. 

14. 

142 

14. 

II. 

COMMISSION'S  REPONSE 

The  Commission  welcomes  this  independent  report  and  appreciates  the  effort  which  had  to  be  made  by 
Panel  members  to  come  to  grips  with  such  a  complex  programme  in  such  a  short  period.  The  report 
underlines  the  importance  of  a high  level  of  international  S&T  activity  to  strengthen  the  S&T  basis  of  EU 
industry  and  to promote  all the  research  actions  necessary  to fulfil  Community  policies. 

The  Panel  recognizes  the  importance  of  the  INCO  Programme  in  providing  coherence  and  greater 
visibility  to  EU  S&T  international  cooperation,  and  points  out  the  particular  contribution  which  can  be 
made  by  INCO  to  the  solution  of  problems  of  a  regional  dimension  which  are  not  normally  tackled  by 
bilateral  programmes  or other  specific  programmes. 

The  Commission  agrees  on  the  potential  of  COST  for  future  expansion,  its  high  relevance  to  EU  policy 
and  beneficial  inter-action  with  the  Framework  Programme.  Any  increase  in  resources  will  however  need 
to  be  carefully  balanced  with  the  needs  of  other  parts  of  the  Framework  Programme.  A  High  Level 
Reflection  Group  has  been  constituted  to  consider  the  future  role  of  COST  in  the  European  S&T 
landscape.  Maximum  opportunity  will  be  taken  to  simplify  and  stream-line  the  organizational  and 
administrative  procedures,  consistent  with  the Community's  financial  regulations.  The detailed  measures  to 
be  worked  out  will  draw  substantially  on  the  results  of  the  COST  evaluation  now  taking  place,  in  relation 
with  the  COST  Senior  Officials  and  the COST  Ministerial  Conference  in  May  1997. 

Concerning  the  suggestion  of  a  pilot  scheme  providing  fast-track  funding  for  Community  participation  in 
Eureka  projects,  any  reservation  of  funds  specifically  for  EUREKA  projects  does  not  seem  realistic. 
However,  the  Commission  intends  to  follow  up  the  suggestion  to  explore  means  whereby  urgent 
EUREKA,  and  of course  other  relevant  projects  could  nevertheless  be  catered  for,  as  has been  done  in  the 
past  (e.g.  COSINE,  JESSI). 

The  Commission  considers  that  a  clear  point  of  reference  responsible  for  international  organizations  is 
needed  to deal  with  the multitude  of  S&T  issues  regarding  them 

With  respect  to  Cooperation  with  Central  and  Eastern  European  countries,  the  Commission  agrees  that, 
particularly  in  the  case  of  the  pre-adhesion  countries,  full  participation  in  Activity  1 programmes  is  the 
ideal  situation  and  should  indeed  be  encouraged,  it  should  nevertheless  be  recognized  that  participation  in 
these  programmes  through  FP4  has  been  very  low  so  far.  Conversely,  the  oversubscription  to  the  INCO-
COPERNICUS  programme  has  been  in  the  order  of  10  to  1.  This  is  due  to  some  extent  to  the  different 
funding  regimes  of  Activity  1 programmes  and  COPERNICUS,  however  it  also  indicates  that  while  there 
is  a  great  desire  for  cooperation  between  Eastern  and  Western  scientists,  there  remains  a  strong 
requirement  for  a dedicated  programme.  The  Panel  recommended  that  COPERNICUS  be  phased  out  with 
the end  of  FP4  and  that participation  in Activity  1 be encouraged  "whenever  this has  a realistic  basis".  For 
most  CEECs,  however,  such  a realistic  basis does not yet exist  and  full  participation  will  only  gradually  be 
achieved.  The  "accompanying  measures"  recommended  in point  5 would  go  some  way  towards  filling  the 
gaps  left  by  COPERNICUS  but  probably  not  far  enough.  It  should  also  be  remembered  that  one  of  the 
important  roles  of  INCO,  recognized  by  the Panel  itself,  is its contribution  to the solution  of problems  of a 
regional  nature,  problems  which  are  not  specifically  addressed  by  Activity  1  programmes.  In preparing  for. 
FP5,  the  Commission  intends  to  increase  the  focus  of  the  specific  RTD  cooperation  with  CEECs  to 
concentrate  on  the  issues  of  particular  relevance  to  these  countries,  as  well  as  placing  more  emphasis  on 
accompaning  measures,  as suggested  by  the  Panel. 

The  Commission  does  not  share  the  view  that  the  role  previously  played  by  INCO-COPERNICUS  in  the 
NIS  should  be  completely  taken  over  by  INTAS,  including  the  aspect  of  financing  the  participations  in 
Activity  1.  The  role  played  by  INTAS  in  S&T  cooperation  with  the  NIS  is  an  important  one,  but  we  do 
not  consider  that  extending  it  further  would  lead  to  any  improvement  of  the  situation  concerning 
participations  in  Activity  1.  To  a  certain  extent,  INTAS  and  COPERNICUS  address  different  types  of 
activity.  While  INTAS  is  mainly  devoted  to  fundamental  research,  COPERNICUS  has  a  more  direct 

143 

14. 

approach  towards  application  oriented  and  industrial  research.  For  the  management  of  financing 
participation  in  Activity  1 programmes,  no  special  agencies  would  be  needed  as this  is  already  established 
within  the  Commission  services  of  the  relevant  programmes.  In  the  preparation  of  FP5,  it  is  intended  to 
look  closely 
in  certain  CEC/NIS,  as 
recommended  by  the  Panel.  The  Commission  agrees  with  the  importance  placed  by  the  Panel  on the  close 
cooperation  necessary  between  PHARE/TACIS  and  INCO  and  it  is  intended  that  a joint  communication 
(DGIA-DGXII) on  S&T cooperation  with the pre-adhesion  countries be produced  in  1997. 

into  the  possibilities  for  support  of  Centres  of  Excellence 

The  Commission  notes  the  support  of  the Panel  for  its policies  in negotiating  S&T cooperation  agreements 
with  non-European  industrialized  countries  including  Emerging  Economies  and  shares  the  Panel's  view 
that  reciprocity  and  suitable  IPR arrangements  should be established  and that  collaboration  be  implemented 
through  the  specific  programmes  on  a  project-by-project  basis.  The  general  need  to  review  the  structure 
and  operation  of  the  Japan  S&T  fellowship  programme  is  acknowledged.  However,  the  proposed  phasing 
out  of  the  EU  fully  financed  fellowships  would  result  in  eliminating  the  access  they  provide  to  Japanese 
industrial  laboratories,  since  the  other  schemes,  which  involve  Japanese  funding,  are  restricted  to  public 
research  organizations.  The  Commission  welcomes  the  suggestion  of  integrating  into  a  more  general 
fellowship  programme  the  Korean  and  any  new  geographically  oriented  fellowship  scheme,  which  may  be 
instituted  with  countries  in Area B. 

With  respect  to cooperation  with  Developing  Countries  (Area  C),  the  Panel  has underlined  the  importance 
of  this  part  of  the  programme  and  recognized  that  the  major  development  areas  of  health,  agriculture  and 
environment  should  be  maintained.  The  Commission  shares  the  opinion  that  applied  research  in  the 
technology  area  is  important  and  that  additional  funds  will  be  needed  in  order  to  adequately  tackle  this 
issue.  As  a  first  step,  it  is  intended  to  increase  the  funding  available  to  "Additional  sectors  of  mutual 
interest"  in the  INCO-DC  programme  of  FP4  from  20 MECU  to  30 MECU.  Specific  cooperation  policies 
are  also  being  developed  for  those  Developing  Countries  which,  due  to  their  economic  and  technological 
growth,  should  be  considered  as  emerging  economies.  While  the  Commission  appreciates  the  large 
number  of very  highly  rated  proposals  submitted  in the  context  of INCO-DC,  it recognizes  the  necessity  to 
reduce  the oversubscription  and  thus the wasted  effort  on behalf  of the scientific  community. 

Concerning  the  recommendations  relevant  to  all  programmes,  the  Commission  agrees  that  it  is  necessary 
to  constantly  make  efforts  to  reduce  perceived  "procedural  inefficiencies"  leading  to  long  delays  between 
submission  of  proposal  and  the  first  transfer  of  funds  to  the  successful  applicants  and  to  ensure  that 
unsuccessful  applicants  are  informed  rapidly  of  their  situation.  Continuous  attempts  are  being  made  to 
ameliorate 
remaining  consistent  with 
Commission's  duty  of public  accountability. 

internal  Commission  administrative  procedures  while 

the 

The  electronic  submission  of  proposals  has  already  been  tried  by  some  programmes  in  the  past.  The 
possibilities  could  be  examined  again  in  the  light  of  updated  electronic  methodologies.  The  facilities 
available  in relevant  countries  would  need to be  investigated  and to be taken  into  account. 

The  Commission  fully  agrees  with  the  recommendation  that  an  integrated  Management  System  be 
introduced.  Although  the  suggestion  is  relevant  to  all  programmes,  it  is  particularly  so  in  the  case  of 
INCO,  where  several  different  DGs are  involved  in its management.  It  is intended  to improve  this  situation 
as  far  as  possible  over  the  remainder  of  FP4,  and  particularly  to  have  a  system  in  place  before  the 
commencement  of FP5. 

Concerning  the  role  and  mandate  of  the  INCO  Programme  Committee,  it  is  governed  by  the  Comitology 
Decision  of  1987 and  Article  130i of the Maastricht  Treaty. 

144 

14. 

III. 

EVALUATION  PANEL 

Mr Luis Avelar 
Director,  Portugal  Telecom 

Prof.  Niels Ole  Bernsen 
Center  for  Cognitive  Science 
Roskilde  University 
Denmark 

Dr.Stein W Bie 
Director,  Research,  Extension  and Training  Division, 
Sustainable  Development  Department 
Food  and  Agriculture  Organization  of the  United  Nations 

Mr Christian  Dambrine 
Association  Nationale  de  la Recherche  Technique,  France 
Member  of the  Bureau,  European  S&T Assembly 

Prof. Dervilla Donnelly 
University  College  Dublin,  Ireland 
Vice-President,  European  S&T  Assembly 

Dr Désirée Edmar 
Director,  Naturhistoriska  Riksmuseet 
Swedish  Museum  of Natural  History,  Stockholm 

Mr Terence  Garrett 
Formerly  Department  of Trade and  Industry 
and the  Royal  Society,  UK 

Dr Werner  Menden 
Formerly  Bundersministerium  fur  Forschung  und 
Technologie,  Germany 
Member  of the  1995 Monitoring  Panel  for  the INCO  Programme 

Dr.Alberto  Silvani 
Consiglio  Nazionale  delle  Ricerce 
Italy 

145 

15. 

1996 FIVE-YEAR  ASSESSMENT 

DISSEMINATION AND OPTIMIZATION OF RESULTS 

(INNOVATION) 

147 

L 

1. 

2. 

3. 

4. 

5. 

6. 

15. 

EXECUTIVE  SUMMARY 

Among  European  decision-makers,  there  is  general  agreement  that  frequent  and  systematic 
innovation  is a main  source  of  new jobs,  high  productivity,  and,  in the  end,  of  wealth  creation.  But 
innovation  needs  a  favourable  environment.  This  means  management  skills,  circulation  of 
knowledge  across  countries  and  sectors,  flexible  product  markets,  and  market-oriented  RTD.  It  also 
means  standards  and  regulations  that  promote  innovation;  a  tax  policy  conducive  to  innovation; 
conditions  favouring  the development  of capital  markets  for  financing  innovation;  a European  patent 
policy  that  facilitates  access  to  intellectual  property  rights  and  defends  those  property  rights  more 
effectively  worldwide.  Finally,  much  more  than  is  now  the  case,  research  institutions  and  industry 
should  work together  to meet the needs expressed  on the market by users  and  consumers. 
INNOVATION,  the  specific  programme  corresponding  to  the  Third  Activity  of  Framework 
Programme  IV,  deals  with  most  of  this.  INNOVATION  became  operational  following  the  Council 
Decision  94/917/EC  of  15 December  1994.  The  programme  has  a budget  of  312  MECU.  It  carries 
on  activities  started  under  SPRINT  and  VALUE  II,  and  one  action  line  from  THERMIE.  Its  basic 
mission  is  to  strengthen  the  innovative  capacity  of  European  industry.  The  programme  must  also 
contribute  to  economic  and  social  cohesion.  There  is  a  special  emphasis  on  SMEs  and  technology 
diffusion  and  circulation.  All  the  programme  must  be  conducted  under  the  principle  of  subsidiarity: 
it must  complement,  but not be a substitute  for,  national  and  regional  activities  in similar  domains. 
The  INNOVATION  programme  is  evolutionary  and  experimental.  It  is  particularly  important, 
therefore,  that  it  be  regularly  refocused  by  proper  monitoring  and  evaluation  based  on  verifiable 
objectives.  The time and  resources  available  for  this  report,  however,  have  dictated  that  it be  more a 
review  on  the  basis  of  relevant  documentation,  some  interviews,  and  our  general  knowledge  of  the 
area,  rather  than  an  evaluation  with  strong  empirical  backing.  The  Commission  should  adopt 
measures  to ensure more  effective  evaluation  in  future. 
The  Terms  of  Reference  for  this  assessment  included  five  questions.  We  were  asked  to judge  the 
relevance,  effectiveness,  and  cost  efficiency  of  the  initiatives  carried  out  within  the  domains  of  the 
INNOVATION  programme  in  the  years  1991  to  1995.  Moreover,  having  in  view  the  socio 
economic  conditions  expected  pver  the  five  years  to  come,  we  had  to  make  recommendations  about 
the  future  orientation  and  management  focus  of  the  programme.  The  assessment  had  to  take  into 
account  the  final  evaluations  of  VALUE  II,  which  was  part  of  Framework  Programme  III,  and  of 
SPRINT,  which was outside the  Framework  Programme. 
Our  overall  impression  is  positive.  INNOVATION  and  its  precursors  SPRINT  and  VALUE  II 
have  addressed  issues  relevant  to  the  Community's  socio-economic  needs.  These  programmes  have 
increased  the  understanding  of  the  innovation  processes  and  policies  within  the  Commission  and 
among  national  and  regional  actors.  The  merger  of  SPRINT  and  VALUE  II  was  a necessary  step to 
rationalize  previously  separated  streams  of  action.  There  are  some  priorities  to  reassess  and  some 
shortcomings  to fix,  but these problems  are not  paramount. 
In particular,  INNOVATION  and its precursors have: 
• Set up  infrastructures  to disseminate  information  on the Community  RTD activities; 
• Learned  and disseminated  useful  lessons on the  innovation  processes  in SMEs; 
• Promoted  horizontal  actions  within  the  Commission  to  make  RTD  and  other  Community  policies 
more  innovation  oriented.  Regional  actions  are the most  successful  example; 
• Created  a seed  for  a European  innovation policy.  The  Green  Paper  on Innovation  should  get  much 
of the credit  for  this; 
• Recognized  that action at Community  level  should  complement  national  and  regional  activities,  not 
be a substitute  for  these. 

7.  We believe  that  innovation  policy  should have an enhanced  role  in EU  activities.  We  emphasize 
in our  report  the  importance  of  local  connections  and  even  personal  contacts  in diffusing  innovative 
ideas.  We  believe,  however,  that  there  is  a  place  for  Community  action,  provided  that  excessive 
centralization  is  avoided.  The  action  plan  based  on  the  Green  Paper  on  innovation  should  lead  to  a 
European  innovation  policy  with  a distinct  and  strong  role  among  the  other  EU  policies  and  among 
national  and  regional  innovation  policies.  This  policy  should  be  supported  by  new  organizational 
arrangements  within the  Commission. 

149 

15. 

8. 

9. 

In  the  light  of  this,  it  would  be  right  to  give  INNOVATION  a  strategic  and  more  influential 
role.  INNOVATION  has been  hampered  by having  too  few  resources  and  too  little  authority.  It has 
nevertheless  made progress  in the direction  of a European  innovation policy.  It therefore  deserves  to 
be  enhanced. 
INNOVATION  should  be  a  wide-ranging  and  flexible  entity  within  the  Framework 
Programme.  There  is a complex  relationship  between  RTD  and  innovation.  Each  contributes  to  the 
other,  but  innovation  is a wider  concept  and  has  other  roots,  besides  those  in RTD.  On balance,  we 
feel  it  is  right  to  keep  INNOVATION  within  the  Framework  Programme.  The  Framework 
Programme,  however,  must  not  unduly  constrain  INNOVATION  from  its  wider  activities.  This 
means  developing  a  complex  of  horizontal  interactions  and  may  need  a  rather  more  open  structure 
than  is appropriate  for  the RTD programmes.  As we mentioned  before,  the programme  must be  able 
to  experiment  and  evolve.  This,  in turn,  means  there  must  be  well  defined  procedures  for  initiating 
and  terminating  activities,  i.e.  'entry  and  exit  mechanisms'.  This  also  implies  that  the  programme 
must  be  able to respond  to needs that  change  rapidly.  Flexibility  should be  increased. 

10.  We  would  like  to  see  INNOVATION  and  its  staff  acting  as  a  'ginger  group'  within  the 
Commission  as  well  as  outside  it.  A  'ginger  group'  is  a  small  group  of  people  whose  role  is  to 
provoke  and  stimulate  activity,  to encourage  new  thinking  and  hence  to enliven  a larger  group.  This 
is  precisely  the  part  we  think  those  involved  with  the  programme  could  and  should  play.  By 
disseminating  ideas,  new  experience,  and  the  results  of  experimentation,  they  would  provoke  and 
stimulate  innovative  approaches.  This  would  give  the  programme  staff  something  of  the  nature  of  a 
think-tank  for  pursuing  the  aims  of  the  Green  Paper.  They  would  need  to  maintain  a  clear  view  of 
how  innovation  works,  provide  a  strong  element  of  reflection  and  experimentation,  and  have  the 
main  objective  of promoting  innovation  inside and  outside  the  Commission. 

12. 

11.  We  recommend  strengthening  the  programme's  coordination  with  other  EU  policies  and  DGs. 
Interfacing  with,  and  contributing  to  the  policies  of,  other  DGs  would  be  a  vital  element  of  this 
'ginger  group'  role.  In  particular,  the  programme  should  be  linked  with  the  assessment  and 
implementation  of  activities  under  Community  policies  other  than  RTD,  such  as  the  industrial, 
SME,  and  regional  policies:  all this to help  innovation,  as well  as technology  transfer  and  diffusion. 
These  matters  require  a committed  advocate, but  also proper  administrative  arrangements. 
It is particularly  important to enhance the interaction  between the  Third  Activity  and the  RTD 
programmes  of the First Activity.  In our view,  the INNOVATION  programme's  mandate  does not 
distinguish  adequately  between  exploiting  the  results  of  specific  RTD  projects  and  promoting 
innovative  thinking  and  practices  more  widely.  Within  Community-supported  research,  we  think 
there  should  be  greater  attention  to  exploitation,  but  this  should  be  the  responsibility,  primarily,  of 
the  RTD  programme  involved.  INNOVATION,  however,  should  not be  a mere  downstream  service 
available  to  the  RTD  programmes  on  request.  It  should  provide  them  with  guidelines  for 
dissemination  and  optimization  and  play  an  interactive  role  of  monitor  and  mentor  throughout  their 
whole  life.  In  general,  it  should  increase  the  orientation  of  these  programmes  towards  innovation. 
INNOVATION  has  launched  a  number  of  specific  actions  to  ensure  a  coordinated  approach  to 
dissemination  and  optimization.  When  fully  operational,  these  initiatives  could  make  good  progress 
in the  direction  of  effective  coordination.  INNOVATION  should  have  the  authority  and  institutional 
instruments  necessary  to ensure that this occurs. 

13.  The  programme  will  need  to  be  clearer  about  its  priorities.  It  will  also  need  a  simpler 
structure.  All  this  to  give  a  sharper  focus  on  some  issues,  increase  flexibility,  and  improve 
internal  coordination.  We draw attention to the following  points: 

13.1  The  European  Innovation  Monitoring  System  (EIMS)  should  be  a particularly  important  part  of  the 
INNOVATION  programme  facilitating,  as  it  already  does,  the  integration  of  Community  activity 
with  that  of  member  states  and  other  organizations.  It  could  play  an  important  role  within  the 
'ginger  group'  activities we advocated  above for  the programme. 

13.2  The  programme  should  expand  and  intensify  its  regional  actions.  In  particular,  using  existing 
initiatives  such  as  the  Regional  Innovation  Strategies  or  new,  more  specific,  instruments,  the 
programme  should  help  eligible  regions  to make  efficient  use  of  structural  funds  in domains  affected 
by  technological 
innovation  and  should  encourage  SMEs  in  these  regions  to  participate  in 
cooperative  innovation  projects. 

150 

15. 

13.3  The INNOVATION  Relay  Centres  (IRCs)  are  in the best position  to  integrate  action  at the  local  and 
European  level.  The  IRCs  should  strengthen  this  as  their  prime  role  within  the  programme.  The 
IRCs  should  work  on  promoting  innovation  in  their  local  environments  rather  than  concentrate  on 
disseminating  information  about  Community  RTD.  Whenever  useful,  they  should  give  a  European 
orientation  to  the  demand  for  innovation  and  innovation  financing  of  local  firms.  They  could  also 
provide  valuable  services  to  other  parts  of  the  programme,  such  as  the  regional  actions.  To  do  this 
they  need  to  be  aware  of  all  aspects  of  innovative  thinking  and  to  be  very  close  to  their  local 
economies,  building  on,  but  not  duplicating,  other  locally  based  activities.  They  also  need  to  be 
close to the  Commission  and  to other  IRCs so that they  can operate  as an effective  network. 

13.4  Creating  networks  is in our view an important  role for  innovation policy  at  European  level.  Through 
these,  information  and  ideas  will  flow,  enabling  cross-fertilization  of  regional  and  sectoral 
experience  to  occur  and  stimulating  competitive  advances.  Such  networks  will  sometimes  be 
electronic,  sometimes  paper-based,  and  often  working  through  personal  contact.  They  may  be  either 
permanent,  i.e.  infrastructural,  elements  of  policy,  or  experimental  initiatives  meant  to  continue  on 
their  own.  Within  the  INNOVATION  programme,  networking  initiatives  other  than  the  IRCs  should 
be  experimental.  They  should  be designed  to become  in due  course  financially  independent  from  the 
Commission.  These  initiatives  could  add  a  European  dimension  to  networks  that  already  exist  or 
start  cooperation  in some new  areas where  the Commission  has  competence. 

13.5  We  attach  particular  importance  to  stimulating  a  financial  environment  favouring  the  diffusion  of 
new  technologies.  So  far,  however,  the  INNOVATION  programme  has  made  only  limited 
contributions  to  thinking  in  the  field.  It  should  aim  at  a  larger  role.  Europe  has  been  too  slow  in 
developing  systems  for  financing  innovative  activity.  These  need  to  be  developed  in  a  coordinated 
way  to  integrate  actions  at regional,  national,  and  Community  level. 

13.6  The  programme  should  retain  the  important  information-providing  role  which  it  discharges  via 
CORDIS  (preferably,  further  enhanced).  It  should  give  more  attention  to  understanding,  in  depth, 
the  needs  of users  of the  service  so that the  innovation  message  is broadcast  to a receptive  audience. 
13.7  We  strongly  recommend  additional  effort  and  a  more  coherent  strategy  on  patents.  This  means 
primarily  strengthening  the present  role  of support  to the  RTD  programmes.  But  the  development  of 
provisions  for  intellectual  property  rights  is  also  a  strategic  issue  for  a  European  innovation  policy. 
The current  situation  is far  from  satisfactory.  Measures  in this area  should  be  intensified  in  future. 

13.8  The  purpose  of  projects  within  the  INNOVATION  programme  should  be  to  demonstrate  and 
disseminate  innovative  approaches  and  ways  in  which  technology  can  enhance  innovative  attitudes. 
It  should  not  be,  only,  to  exploit  specific  results.  These  projects  should,  primarily,  serve  as 
showcases  for  displaying  innovative  thinking  and  practices  generally.  They  should  aim,  rather  than 
at  specific  technologies,  at  the  wider  objectives  of  a  European  innovation  policy  such  as  an  easier 
access  to product  markets  for  new  technology-based  firms,  the  increase  of  the  technology  absortion 
capabilities  in  traditional  sectors,  a  more  frequent  transfer  of  technologies  between  sectors  and 
countries  (not  only  within  the  Community,  but  also  across  its borders),  the  stimulation  of  a  'culture 
to  innovate'  throughout  Europe. 

13.9  We believe  that  the  transfer  and  validation  projects  should  be  merged,  the  OPET  network  should  be 
transferred  to  THERMIE,  and  two  of  the  present  action  lines,  innovation  management  techniques 
and  public  awareness  of  research  and  technology,  should  not  be  individual  elements  of  a  future 
programme,  although  their  specific  goals  may  be  important.  The  awareness  activities  could  become 
a  strand  of  the  EIMS  and  be  redirected  to  a  coherent  promotional  strategy  of  the  Community 
initiatives  in  research,  technology,  and  innovation.  The  innovation  management  initiatives  could  be 
shared  out:  the  EIMS  would  assess  new  techniques,  whereas  the  IRCs  would  promote  state-of-the-
art  practices. 

14.  The  previous recommendations  may be summarized  in the six points below: 

• Innovation  policy  should have an enhanced  role  in EU  activities. 
• INNOVATION  should  continue,  be  significantly  enhanced,  and  become  a  wide-ranging  and 
flexible  entity  within  the  Framework  Programme.  It  should  provide  a  'ginger  group'  to  stimulate, 
promote,  and  champion  innovation  within the Commission  as well  as outside  it. 
• There  should  be greater  attention  to  innovation  within the RTD  programmes. 

151 

15. 

• INNOVATION  should  assist  the  dissemination  and  optimization  activities  of  the  individual  RTD 
programmes.  It  should  have  the  opportunity  to  intervene  where  these  activities  appear  to  be 
inadequate.  In  general,  it  should  increase  the  orientation  of  the  RTD  programmes  towards 
innovation. Proper administrative arrangements are needed to ensure all this. 
• The  coordination  with  other  EU  policies  and  DGs  should  be  strengthened.  In  particular,  the 
programme should be linked with the assessment and implementation of activities under Community 
policies  other  than  RTD,  such  as  the  industrial,  SME,  and  regional  policies:  all  this  to  help 
innovation, as well as technology transfer  and  diffusion. 
• There  should  be  greater  attention  to ordering  the priorities  of  the programme's  individual  action 
lines  so  as  to  give  a  sharper  focus  on  the  issues  highlighted  in the  previous  points.  In  future  the 
programme would consist of eight major elements. 

152 

15. 

II. 

COMMISSION'S  RESPONSE 

The Commission  fully  supports the emphasis  in the recommendations  on the promotion  of  innovation,  both 
in-and  outside the RTD  Framework  Programme. 

As  a  matter  of  fact  the  Commission  adopted  on  20  November  1996,  following  the  wide  ranging  public 
debate  stimulated  by  the  Green  Paper  on  Innovation,  the  First  Action  Plan  for  Innovation  in  Europe.  This 
Plan  considers  the  ability  to  innovate  as  a necessity  to maintain  competitiveness  and  a precondition  for  the 
future  creation  of jobs  in Europe. 

It  identifies  three  areas  for  action  : to  foster  a  genuine  innovation  culture,  to  establish  a  legal,  regulatory 
and  financial  framework  conducive  to  innovation,  to  better  articulate  research  and  innovation.  Each  of 
those  areas  includes  actions  which  belong  to the  realm  of science,  research  and  technology  policies  as well 
as actions  which typically  belong  to other policies  such as industrial  -,  enterprise  -,  and  regional  policies. 

The Commission  also  agrees  with the  recommendations  that there  should  be greater  attention  to  innovation 
within  the  RTD  programmes  and  that  the  INNOVATION  programme  could  usefully  guide  and  assist  the 
specific  programmes  in  these  efforts.  In  the  light  of  the  experience  of  the  Interprogramme  Group  on 
Dissemination  and  Optimization,  a specific  mandate  will be considered  for  the  INNOVATION  programme 
providing  the  basis  and  instruments  necessary  to  ensure  that  this  mentoring  and  monitoring  effectively 
occurs. 

The  same  applies  as  regards  the  recommendation  on  the  coordination  with  other  EU  policies  with  a  view 
to promote  innovation,  technology  transfer  and technology  diffusion. 

Thirdly,  the  Commission  welcomes the proposed  re-structuring  of the  activities  covered  by  the  programme 
and  the  suggested  reduction  in  the  number  of  action  lines.  In  this  context  it  wants  to  point  out  that  the 
latest  call  for  proposals  for  Technology  Transfer  and  Technology  Validation  Projects 1 already  has  come  a 
long way  towards  integrating  both types of projects  as indicated  by one of the  recommendations. 

Finally  the  Commission  has  taken  notice  of  the  recommendations  that  the  INNOVATION  programme  be 
clearer  about  its  priorities  :  exploiting  the  results  of  specific  RTD  projects  versus  promoting  innovative 
thinking  and  practices  more  widely.  In line with  the orientations  adopted  by  the  Action  Plan  for  Innovation 
the  latter  is to emerge  as one of the  key priorities  for  future  operations 

OJN°C221,  17.09.96, p. 20. 

153 

15. 

III. 

EVALUATION  PANEL 

Luis A.  Oro Giral (E),  Chairman 
Professor,  Department  of Inorganic  Chemistry 
University  of  Zaragoza 

Nicos M.  Christodoulakis  (GR) 
Professor,  Department  of International  Economics  and  European  Studies 
Athens  University  of  Economics  and  Business 

Heinrich  Hôfer  (D) 
Head,  Department  of Technology  and  Innovation  Policy 
Federation  of German  Industries  (BDI,  Bundesverband  der  Deutschen  Industrie) 

Sean McCarthy  (IRL) 
Managing  Director,  Hyperion  Energy  Systems Ltd 

Erkki  Ormala (SF) 
Secretary,  Science  and  Technology  Policy  Council  of  Finland 
Ministry  of Trade  and  Industry 

Philip V.  Hills (UK) 
Professorial  Fellow,  PREST  (Policy  Research  in Engineering,  Science,  and  Technology) 
The  University  of  Manchester 

Giorgio Tuninetti  (I),  Rapporteur 
Assistant  to the  First  Vice-President,  Business  Development 
Corporate  Staff,  Finmeccanica  SpA 

154 

16. 

1996 FIVE-YEAR  ASSESSMENT 

STIMULATION OF TRAINING AND MOBILITY 

OF THE RESEARCHERS 

155 

16. 

I. 

EXECUTIVE  SUMMARY 

This  is  the  Executive  Summary1  of  the  Five  Year  Assessment  of  the  Human  Capital  &  Mobility  (HCM) 
and  its  successor  the  Training  &  Mobility  of  Researchers  (TMR)  Programmes.  The  Assessment  is 
undertaken  as  a  precursor  to  the  adoption  of  the  TMR  Programme  under  the  Fifth  Framework 
Programme.  It also  incorporates  the Final  Evaluation  of the HCM  Programme. 

The  recommendations  made  in this  Report  are  essential  to the  good  development  of  the  TMR  Programme 
and  have  the  total  and  unanimous  support  of  the  Assessment  Panel.  It  is  the  Panel's  explicit  wish  to  see 
these recommendations  fully  implemented. 

Europe,  as  it  enters  the  21st  century,  faces  a  number  of  major  challenges.  Its  industrial  competitiveness 
must  be  improved  and  barriers  to  the  flow  and  development  of  information  and  knowledge  broken  down. 
Ecological  and  environmental  problems  must  be  overcome.  The  major  social  concerns  of  migration  and 
racism,  health  and  lifestyles,  social  exclusion  and  governance  must  be  faced.  The  Panel  concludes  that 
Europe  will  be  better  positioned  to  face  theses  challenges,  if  its  scientific  and  technological  research 
community  is  ready  to  cooperate  across  discipline,  across  culture  and  across  regional  and  national 
boundaries  and  that  a  training  and  mobility  programme  has  a  substantial  contribution  to  make  in 
developing  this  cooperation. 

•  The  TMR  Programme  is  an  essential  element  in  facing  these  challenges  and  should  be  continued 

under the Fifth  Framework  Programme. 

the  Global  dimension  of 

However,  any programme  must  have the  capability  of transcending  purely  EU  concerns  to ensure  research 
industrial  competitiveness  and  of  ecologically  sustainable 
encompasses 
development  and,  in  particular,  plays  a  full  part  in  the  development  and  stabilization  of  Central  and 
Eastern  Europe.  To  respond  fully  to  these  challenges,  some  changes  are  needed  in  what  the  Panel 
considers  to be a most  successful  and  well run  Programme. 

•  The  Programme  needs  to  move  to  working  with  all  research  disciplines  relevant  to  these 

challenges and developing in them a full European and Global  dimension. 

The  Programme  must  also  work  closely  with  European  research  community  itself,  where  it  has  earned 
great  respect  for  its  emphasis  on  scientific  excellence  and  a  "bottom-up"  approach.  The  latter  permits 
dynamic  new  areas  outside  of  the  EU  specific  Programmes  to  be  supported  and  corrects  for  the 
misalignment  or  inflexibility  of highly targeted programmes.  It is in a sense an insurance  policy. 
However,  it  does  cause  major  problems  in  over-subscription  in  parts  of  the  Programme  and  confusion  in 
Programme  objectives  : is it an insurance  policy  for  the research  not covered  by  specific  Programmes  or  is 
it a training  and mobility  programme  ? 

The  Panel  recommends  that : 

•  All  5FP  Programmes  should  have  a  section  for  "bottom-up"  research  -  their  insurance  policy. 

The TMR should remain bottom-up but concentrate on training and mobility issues. 

Turning  to the  individual  Programme  Activities : 
•  The Fellowships Activity  should be developed  as three basic activities.  The Panel  recommends  that : 

1.  The  Marie  Curie  Fellowship  should  be  developed  explicitly  for  the  very  ablest  of  Europe's 

researchers  with a proven  research  record. 

^ h is  Summary  is published  on a stand-alone  basis in all Union  languages  (Document  1) and  as  part 
of the  full  report  in English  only  (Document  2). 

157 

2.  Industrial  Host  Fellowships  should  provide  support  specifically  for  the  training  researchers  in 

industry. 

3.  Structural  Cohesion  Fellowships  should  provide  specifically  for  the  development  of  high  level 

research  teams  in Less  Favoured  Regions. 

16. 

•  The  Research  Networks  Activity  should  increase  the  success  rate  and  value  for  money  of  networks 

by: 

1.  Increasing  the flexibility  of  network  structures  funded,  possibly  moving  away  from  the  average 

funding  per  node basis. 

2.  Providing  a maximum  of 50%  funding  for  projects, 
3.  Increasing  quality  of networks  fellows  by open,  international  advertisement  of positions, 
4.  Encouraging  interdisciplinary  proposals  and ensuring  their  equitable  evaluation  structure. 

•  The Large  Scale Facilities Activity  should : 

1.  Increase  the cluster  development  and  co-ordination  elements  of the  Activity. 

These  have a high  EU value  added. 

2.  Broaden their  definition  to being  "Large  Scale Intellectual  Centres  of  Excellence". 
3.  Avoid becoming  the platform  for  the planning  of  news  LSFs.  This might  become  damaging  to 

the main  objectives  of the Activity. 

4.  Be reviewed  for  repositioning  within the Framework  Programme. 

•  The  Accompanying  Measures  Activity  : There  is a major  need  for  a  "Strategy  &  Follow-up  Unit"  to 
support  the  Activities  of  the  TMR  Programme.  This  Unit  will  have  many  tasks  essential  to  overall 
Programme  development  including  developing  : 

•  An overall  strategic  and  analytic  capacity  to support  the  Programme, 
•  A  working  model  of  the  European  industrial  and  academic  research  community  needed  for 

Programme  planning  and  evaluation, 

•  A  follow-up  capacity  which  will  coordinate  and  analyse 

the 

impact  of  the  TMR 

Programme, 

•  Associated  feedback  mechanisms  to identify  shortcomings  and potential  improvements, 
•  A support  platform  to work  with Member  States on trends  in research  training,  mobility  and 

related  issues. 

158 

II. 

COMMISSION'S  RESPONSE 

1. 
The  Commission  welcomes  the principal  conclusion  of the Panel  that the promotion  of  the  training 
and  mobility  of  researchers  is  an  essential  element  of  the  Fourth  Framework  Programme  and  should  be 
continued  under  the  Fifth  Framework  Programme.  The  Commission  shares  the  Panel's  view  that  the 
current  Training  and  Mobility  of  Researchers  (TMR)  Programme  is  a  most  successful  and  well  run 
Programme. 

2. 
The  Panel  has made  a number  of  specific  recommendations  to  improve  still  further  the  impact  and 
effectiveness  of  these  training  and  mobility  actions,  in  particular  with  a  view  to  the  Fifth  Framework 
Programme.  These  recommendations  will  be  studied  carefully  to  see  how  they  can  best  be  implemented, 
in particular: 

• 

• 

• 

• 

focusing  the  Marie  Curie  Fellowship  scheme  on  the  very  ablest  of  Europe's  young  researchers 
with  a proven  research  experience; 

the  training  of 
introducing  Industry  Host  Fellowships  to  provide  support  specifically 
researchers  in  industry,  as  well  as  Regional  Host  Fellowships  to  improve  the  development  of 
research  and  research  training  in the  less-favoured  regions  of the  Community; 

for 

finding  mechanisms  to  increase  the  success  rate  and  value  for  money  of  Research  Training 
Networks; 

introducing  Infrastructure  Networks  as  a  means  of  promoting  self-coordination  among  Europe's 
research  infrastructure; 

•  organising  a  Strategy  and  Follow-up  Group  to  provide  strategic  and  analytic  support  for  the 

activities. 

The  Commission  shares  the Panel's  view  that  these  actions  should  remain  "bottom-up",  open  to all 

3. 
fields  of  scientific  research,  and  with projects  selected  on the basis  of  scientific  excellence. 

159 

16. 

III. 

EVALUATION  PANEL 

Professor  Daniel  Thomas  (Chairman) 
Professor  at the  University  of Technology  of Compiègne  (France).  Since  1970, he has held  responsibilities 
at  a national  and  European  level.  He  heads  the  enzyme  technology  laboratory  belonging  to the  CNRS.  He 
has published  over  300 scientific  papers  in international journals.  For  several  years, he  was  Director  of the 
French  Biotechnology  Programme.  At  a  European  level,  he  initiated  the  Biotechnology  activities  in  1978 
and has chaired  many  E.C.  committees. 

Professor  Joannes  (Jan)  Beenakker 
Emeritus  Professor  of  Physics  at  Leiden  University.  Before  his  retirement,  he  headed  the  molecular 
physics  group  of the  Department  of  Physics  at  Leiden  University.  His main  field  of  research  over  the  past 
decade  has  been  the  study  of  nonequilibrium  behaviour  of  gases  of  rotating  molecules.  He  has  been  guest 
Professor  of  molecular  physics  at the  Catholic  University  of  Leuven.  From  1985 until  1991 he  was  Rector 
Magnificus  of  Leiden  University.  After  his  retirement,  he  served  as  Chairman  of  F.O.M.  (the  physics 
section  of  the  Dutch  Science  Foundation  NWO),  from  1991  until  1994.  He  is  a  member  of  the  KNAW, 
the Royal  Dutch Academy  of Science  and of the Academia  Europea. 

Mr.  Tom  Casey  (Rapporteur) 
Director,  human  Resources,  at the  CIRCA  Group  Europe  Ltd.  and  a lecturer  in Technology  &  Innovation 
Policy  in  University  College  Dublin.  He  was  educated  at  Imperial  College,  University  of  Manchester  and 
Université  de  Paris-Dauphine.  He  has  worked  in  the  OECD,  National  Board  for  Science  &  Technology, 
National  Enterprise  Agency  in Dublin. 

Professor  Peter Hackl 
Professor  of  Statistics  at  the  University  of  Economics  and  Business  Administration,  Vienna,  Austria.  His 
scientific  interests  concern  the  development  and  applications  of  statistical  methods  for  time  series  analysis 
and  for  the  analysis  of  economic  data  e.g.  diagnostics  for  economic  and  econometric  models  and  methods 
for  process  control  and  for  quality  improvement.  He  is  author  of  several  books  and  some  70  articles  in 
refereed  international  journals.  Since  1995,  Professor  Hackl  has  been  President  of  the  Austrian  Statistical 
Society. 

Dr.  Geoff  Robinson 
Director  of  Technology  for  IBM  United  Kingdom.  He  has  spent  most  of  his  career  in  Research  and 
Development  for  IBM.  From  1992 to  1994 he was  Chief  Scientific  Advisor  on  Science  and  Technology  at 
the  UK  Department  of  Trade  and  Industry.  He  is a Fellow  of the  Royal  Academy  of  Engineering  and  was 
President  of the British  Computer  Society  1995-96. 

Professor  Alexandre  Quintanilha 
Holds  a  PhD  is theoretical  Physics  and  is Professor  of  Biophysics  and  Dean  of  the  Biomedical  Institute  of 
the  University  of  Porto.  From  1972 to  1990,  Professor  Quintanilha  was Associate  Professor  of  Physiology 
at  the  University  of  California  at  Berkeley  and  Director  for  Environmental  Studies  at  the  Lawrence 
Berkeley  Laboratory.  His  area  of  scientific  research  is  molecular  markers  and  mechanisms  of  adaptation 
under  stress  in humans. 

160 

17. 

1996 FIVE-YEAR ASSESSMENT 

NUCLEAR FISSION SAFETY 

161 

17. 

T. 

EXECUTIVE  SUMMARY 

The main  impact  of the  programme  has been,  and  continues  to be,  the provision  of  a common  scientific 
basis  for: 

the safe  operation  of  nuclear  reactors, 
their  effective  decommissioning, 
the development  of agreed  means for  the safe  management  of  radioactive  waste,  and 
the  protection  of  the  European  population  and  environment  from  possible  adverse  effects  of 
radiation,  and  in particular  the past,  present  or  future  use of nuclear  power  and  the  uses  of  radiation 
in medicine. 

In reviewing the achievements  of the Third Framework  Programme  (FP3)  we have  concentrated  our 
detailed  attention  on  the  Reactor  Safety  and  Waste  Management  areas,  since  the  Radiological  Protection 
and  Decommissioning  areas have been  subjected  to independent  evaluation during  this five-year  period. 

In  reactor  safety,  new  understanding  has  been  developed  of  phenomena  which  may  occur  in  a  severe 
accident.  Progress  was  made  in  the  area  of  core  degradation,  but  uncertainties  remain  in  the  so-called 
late  phase. 
Progress  was  achieved  in  calculating  distribution  and  combustion  of  hydrogen,  and  basic 
knowledge  was  obtained  in the  area  of  molten  fuel-coolant  interaction.  Several  interesting  core  retention 
Fission  product  behaviour  was  studied  effectively.  Accident  management 
devices  were  developed. 
support  was  initiated  to be available  to help  in recovery  following  a severe  accident. 

In  radioactive  waste  management,  there  has  been  cost-effective  use  of  expensive  underground 
laboratories  both  to  develop  confidence  in  safety  performance  assessment  for  the  selection  of  future 
suitable  sites  which  will  meet  national  criteria,  and  to  understand  mechanisms  of  migration  of 
radionuclides,  many  of  which  are  relevant  also  to  other  toxic  substances. 
There  has  been  useful 
innovation  in  attempting  to  reduce  the  volume  of  various  wastes,  the  content  of  long-lived  alpha-emitting 
materials  to  be  disposed  of  in  repositories,  and  in  the  development  of  waste  forms  and  containers  which 
are  stable  in  the  long  term.  A  'European  Network  for  Quality  Checking  of  Waste  Packages'  has  been 
successful  in providing  harmonization  of  quality  assurance  for  radioactive  waste  products  which  has  been 
of help to national  regulatory  and  licensing  organizations. 

Research  in  decommissioning  has  decreased  as the  technology  has  matured,  and  the  publication  of  the 
Handbook  on  Decommissioning  of  Nuclear  Installations  has  proved  an  excellent  means  of  disseminating 
results  and  of  stimulating  their practical  application. 

in  understanding  DNA  repair  after 

In  radiation  protection,  the  major  achievement  has  been  in  providing  the  scientific  basis  for  the 
Fundamental 
revision  of  the  EC  Basic  Safety  Standards  Directive,  which  was  adopted  in  May  1996. 
in  radiation 
advances 
carcinogenesis,  and  in  epidemiology  have  come  from  the  programme. 
It  has  also  helped  to  evaluate 
patient  benefit  from  conventional  and  newly-introduced  radiological  techniques  and  contributed  to  the 
revision  of  the  Medical  Exposure  Directive.  Specific  achievements  in  retrospective  radiation  dosimetry 
have  also  resulted  from  the programme.  The  socioeconomic  value  of  the  programme  can  be  evaluated  in 
terms  of  life  prolongation  and  improved  quality  of  life  through  better  medical  care,  or  averted  medical 
cost due to avoidance  of adverse  radiation  effects. 

radiation  damage, 

initial  events 

the 

The  analysis  of  the  ecological  consequences  of  the  Chernobyl  accident  has  thrown  into  relief  which 
remedial  actions  are  beneficial  and  which  are  less  so.  A new  approach  to  radioecological  studies  has  led 
to  the  development  of  models  which  may  contribute  to  a  comprehensive  environmental  strategy  for 
response  to any  future  accident.  Finally,  much  has been  achieved  in developing  and  disseminating  within 
the EU  computer  programmes  for  risk  assessment  and decision  support  following  a nuclear  emergency. 

163 

17. 

The transition from  FP3 to the Fourth Framework  Programme  (FP4),  with a smaller  number  of  well-
increased  multi-national 
targeted  research  areas, 
cooperation,  increased  participation  by  industry  and  by  small  and  medium-sized  enterprises  has  led  to 
improved  management  of  what  was  already  an  effective  and  productive  European  research  effort. 
In 
several  areas  of  this programme,  the  work  of  European  organisations  represents  the  leading  edge  of  world 
science,  and these  areas  do represent  real  opportunities  for  added  economic value to European  industry. 

increased  numbers  of  partners 

in  each  project, 

An  important  outcome  of  this  EU-wide  research  and  training  programme  is the  maintaining  of  scientific 
and  technical  competence  even  in those  EU  member  states  where  such  knowledge  is not  provided  as  a by 
product  of  operating  nuclear  power  plants.  The  continuing  safety  of  populations  can  be  assured  only  if 
sufficient  knowledge  is  available  in  all  countries.  This  may  require  that  a  new  generation  of  suitably 
qualified  scientists  emerges  to replace those  who  are  retiring.  Active participation  in future  well-targeted 
EU  research  programmes  is an excellent  means for  fostering  a new generation  of  specialists. 

The  common  scientific  base,  to  which  this  research  programme  has  made  significant  contributions, 
allows  EU  member  states  to  make  their  own  policy  decisions,  but  helps  to  ensure  that  the  logic  by  which 
those decisions  evolve  is  apparent. 

The major  method  of communication  of results  obtained  in this programme  is via publication  in  refereed 
scientific journals,  but publications  by the Commission  have made a useful  contribution  in dissemination  of 
European  and  wider  international  scientific  audiences  have  been 
information  about  the  programme. 
reached  successfully  by  the  numerous  symposia,  workshops  and  conferences  organised  within  the 
programme.  However,  the  use  of  the  results  of  this programme  by  other  Directorates  of  the  Commisson, 
e.g.  in  setting  standards,  in  improving  public  confidence,  and  in  assistance  to  countries  in  central  and 
eastern  Europe,  has been  less than we would  have  hoped. 

The  development  of  FP4  represents  a significant  increase  in  detailed  specification  by  the  Commission 
of  the  types  of  studies  which  would  be  supported,  although  the  programme  evolved  from  wide  discussions 
It  is  important  that  a  balance  is  maintained  to  ensure  that 
to  allow  input  from  individual  scientists. 
opportunity  exists  for  innovative  scientific  groups  in member  states  to obtain  EU  funding  for  work  in new 
or  unorthodox  areas  where  the  chance  of  success  may  be  low  but  the  potential  value  of  success  is 
disproportionately  high. 

FP4  provides  an  excellent  base  for  defining  needs  for  the  Fifth  Framework  Programme  (FP5). 
However,  there  will  be  advantage  in  reviewing  objectives  and  time  schedules  in  view  of  end-user  needs. 
End-users  of  each  of  the  programmes  should  be  identified,  and  links  established  between  the  research 
teams  and  the  users.  Several  results  from  the  programme  can  be  tailored  so  that  they  are  used  by  small 
and  medium-sized  enterprises,  e.g.  in radiation  control,  reactor  safety,  waste  management  and  emergency 
preparedness. 

By  concentrating  further  research  on  risk  relevant  items,  extensive  knowledge  about  severe  accidents 
should  be  available  by  the  end  of  FP5,  providing  a  common  scientific  basis  of  understanding  at  the 
European  level  about  conceivable  reactor  accidents  and  corrective  measures.  This  knowledge  can  be 
utilized  in appropriate  accident  management  procedures  and,  where  applicable,  design  features. 

Underlying  knowledge  for  waste  repositories  should  improve  up  to  the  time  of  actual  disposal.  The 
main  task  is  now  to  show  that  all  the  necessary  studies  and  experimental  investigations  are  carried  out  to 
achieve,  to the  required  degree,  the  safe  performance  of  the  entire  waste  management  system  -  especially 
the  geological  disposal.  In this  context,  the  long  term  tasks  require  further  effort,  not  least  because  of  the 
wide  range  of potential  geological  formations  in the EU member  states. 

We  consider  that  no  further  research  work  will  be  required  on  decommissioning,  since  the  current 
programme  in  FP4  will  lead  to  demonstration  of  a  comprehensive  range  of  successful  techniques  on  a 
wide  range  of  reactor  stations  and  other  nuclear  installations. 

164 

17. 

Study  of human protection  against  radiation  resulting  from  nuclear power  activities  requires  continued 
research.  This area is important in demonstrating improving quality of life for  European populations, and 
It  is  an  area  of  scientific  excellence  and  exceptionally  broad 
relieving  them  of  some  of  their  fears. 
European collaboration,  where the activities of the Commission can guide and ensure enhanced  output of 
national research.  Even if some aspects of the programme in the radiation protection area are of no direct 
relevance to nuclear  safety,  they are of great significance  to the protection  of the patient  in medicine and 
should be continued. 

We  consider  that  the  area  'Mastering  Events  of  the  Past'  can  be  merged  into  the  research  areas 

mentioned above. 

Detailed  recommendations  at the  end  of  the main text  include  suggestions  for  further  improving  the 
Commission's  administrative  procedures,  so as to  leave  more time  for  the  contractors  to concentrate  on 
their research. 

165 

17. 

II. 

COMMISSION'S  R E S P O N SE 

The  Commission  acknowledges  the  work  of  the  independent  evaluation  experts  who  were  confronted  with 
areas  on  Reactor  Safety, 
the  task  to  evaluate  for  the  first  time  the  Euratom  research  and  training 
Decommissioning  and  Radioactive  Waste  Management  and  Disposal 
together  with  the  Radiation 
Protection  Research. 

Since  the  Radiation  Protection  Research  contains  some  subject  fields,  such  as  radiation  from  natural 
sources  and  medical  use  of  radionuclides  and  ionising  radiation,  which  are not  relevant  for  all the  research 
areas  of  the  SP  Nuclear  Fission  Safety,  the  evaluation  panel  had  the  difficult  task  to  situate  the  various 
research  achievements  of the past  as well as to emphasise  research  objectives  for  the  future. 

It  is  in  line  with  the  Commission's  strategy  for  planning  future  research  that  a  more  coherent  approach 
should  be  followed  and  that  the  protection  of the population  and protection  of the  environment  will  remain 
a  main  objective  of  the  NFS  research.  A  further  important  aim  will  be  to  stimulate  the  conservation  of 
expertise  and  industrial  activities  which  assure  European  competitivity  in  the  nuclear  field.  Equally 
important,  when  defining  new  research  objectives,  a greater  emphasis  will  have  to be  put  on  the  end-user 
of the  research  results  and  on the establishment  of  'closure  criteria'  in order  to set more  precise  time  limits 
to the  research  projects. 

The  Commission  concurs  with the specific  conclusions  that  research  on severe  reactor  accidents  should  be 
continued  and  that  innovative  European  reactor  designs  should  be  supported  in the  future.  Furthermore  the 
recommendation  will  be  considered  that  safe  performance  of  the  waste  management  and  in  particular 
geological  disposal  will be of  special  importance  for  this programme. 

The  Commission  shares  the  panel's  opinion  that  an  EU  programme  of  radiation  protection  research  must 
continue  to  ensure  safe  application  of  ionizing  radiation  in  the  different  fields,  as  far  the  public,  the 
workers  and  the  environment  are  concerned.  It  will  thus  help  to  establish  an  objective  framework  to 
prevent  distorted  public  perception  of radiation  risk  from  denying  to Member  States the undoubted  benefits 
of wider  uses  of radiation  in medicine  and  industry,  and  not just  in nuclear  energy. 

The  Commission  underlines  that  one  particular  role  of  its  research  programme  in  NFS  is  to  contribute  to 
establish  a  baseline  for  apparent  logic  in  the  policy  decision  making  of  the  EU  Member  States.  The 
continuing  European  programme  will  help  to  assure  complementarity  and  quality  of  work  across  the 
Member  States. 

to  cooperation  and 
The  Commission  takes  up  the  panel's  view  point  that  more  attention  will  be  paid 
management  aspects, 
in  order  to  improve  the  efficacy  of  the  EC's  effort.  This  includes  that  scientific 
excellence  and  management  skills  will  remain  the  yardsticks  for  future  selection  of  research  priorities. 
The  EC's  research  programme  will  also  be  conceived  to  attract  young  scientists  to  these  fields  of 
research  where  European  competence  risks to loose territory  on the global  scale. 

166 

17. 

III. 

EVALUATION  PANEL 

Mr BENNETT, B.G 
UNSCEAR  Vienna 
Secretary/Committee  on the effects  of atomic radiatons 

Mr ESCHRICH, H. 
retired 
Nuclear reactor  specialist 

Mr  FEATES, F. 
University  of  Manchester 
Professor/environmental  radiation  chemistry 

Mr GUILLAUMONT, R. 
CNRS 
Professor,  Institut de Physique Nucléaire 

Mr  MARCUS,  F.R. 
NKS (Nordic Nuclear  Safety  Research) 
Expert  in nuclear safety  research 

Mr VAYSSIER, G. 
Nuclear  Safety  Consultancy 
Consultant 

Mr WAMBERSIE, A. 
Université Catholique  Louvain  (UCL) 
Professor,  faculté  de médecine, radiobiologie et radioprotection 

Mr BERRY, R.J. 
Consultant 
Nuclear  Safety  Consultant 

167 

18. 

1996 FIVE-YEAR ASSESSMENT 

CONTROLLED THERMONUCLEAR FUSION 

169 

I. 

EXECUTIVE  SUMMARY 

Recommendations 

1.  Fusion  is  one  of  the  few  energy  sources  which  might  make  a  significant  contribution  to  satisfy  the 
growing  need  for  electricity  from  the  middle  of  the  21st  century  onward.  Taking  into  account 
intrinsic  safety  aspects,  potential  environmental  advantages  and  the  wide  availability  of  fuel,  it  is 
important  for  Europe  to  have  this  option  open.  The  long-term  and  the  wide  scope  of  fusion 
development  justify,  in  application  of  the  principle  of  subsidiarity,  the  continuation  of  the  direct 
involvement  of the European  Union  in this R&D area,  and  of the full  integration  of the efforts  of the 
Member  States (and Switzerland)  in a European Fusion Programme. 

2.  The Board  confirms  the validity  of the long-term R&D strategy  recommended  by previous panels and 
endorsed  in the  1994 Council  Decision:  on the path to the Demonstration  reactor  (DEMO),  only  one 
large device  is needed  i.e.  a tokamak experimental  reactor (ITER). 

3.  The Board,  welcoming  the strong  efforts  developed  by the Programme  to focus  on ITER  most  of the 
scientific,  technological  and  industrial  activities,  recommends that this effort  be continued  and  further 
steps be taken to increase the involvement  of industry  in the Programme. 

4.  Fusion  R&D  has  now  reached  a  stage  where  it  is  scientifically  and  technically  possible  to  proceed 
with  the  construction  of  the  first  experimental  reactor,  and  this  is  the  only  realistic  way  forward. 
Starting  the  construction  of  ITER  is therefore  recommended  as  the  first  priority  of  the  Community 
Fusion  Programme under the Fifth  Framework  Programme. 

5. 

ITER  should  be built  in  Europe,  as  this  would  maintain  Europe's  position  as  world  leader  in  fusion 
and  would  be  of  great  advantage  to  European  industry  and  laboratories.  This  would,  however, 
require  an average  increase  of  at  least  50%  in the Community  funding  for  Fusion  in the  first  decade 
of the next century,  accompanied  by the phasing  out of JET,  a redirection  of the Community  funding 
of  the  Associations,  and  a substantial  contribution  from  the  host  country.  For  the  Fifth  Framework 
Programme  the additional  financial  requirement  would be limited to the order  of 200 MECU. 

6. 

If  it turned  out  not to be possible  to have  ITER  in Europe,  the  firm  recommendation  of  the  Board  is 
to maintain a strong participation  in ITER  as the first  priority  of the European  Fusion  Programme. 

7.  The  Board  recommends  that,  with  regard  to  the  construction  of  ITER  :  particular  attention  be 
devoted  to  the  organisation  of  System  Engineering  and  to  the  consequent  optimisation  of  the 
industrial participation;  the contributions by the partners be provided  mainly  in kind;  and a substantial 
fraction  of  the  ITER  staff  be  seconded  by  the  partners  to the project  for  periods  appropriate  to  their 
tasks, and then return to their parent  institutions to optimise the circulation  of know-how  . 

Assuming  this  main  strategy  is  implemented,  and  ITER  is  built,  the  following  further  recommendations 
apply : 

8.  As  soon  as  a  firm  decision  on  the  construction  of  ITER  is  taken,  the  JET  programme  should  be 
reassessed  and  any  possible,  further  activities  concentrated  on  key  issues  for  ITER,  within  the 
stringent  financial  constraints  which  will  be  imposed  by  ITER  construction.  Particular  attention 
should  be  devoted  to  the  management  of  JET  personnel,  as  the  termination  of  the  project 
approaches. 

9  Association  programmes  on both physics  research  and technological  development,  should  be pursued 
with the aim of contributing  to ITER and optimizing  the database  for  DEMO.  Physics  research  to be 
conducted  on  small  and  intermediate  size  devices  in  the  Associations,  should  deal  with  plasma 
physics  and  concept  improvement.  Long-term  technology  work  should  include  the  development  and 
characterisation  of  appropriate  reactor-relevant  materials;  this  will  require  the  construction  of  an 

171 

intense  14MeV neutron source.  The design and development  of the DEMO blanket  should  continue, 
taking full  advantage of the opportunity to test blanket modules in ITER. 

The  Board  supports  the  development  of  the  Stellarator  line,  so  that  future  strategic  decisions  on 
DEMO can be founded  on a sufficiently  broad physics basis. 

The  Board  recommends  that  no  new  initiative  be  launched  on  the  line  of  Reversed  Field  Pinch 
devices as long as no new favourable  data are available. 

New  "clustering"  initiatives between the Associations around  specific  themes,  similar to those  already 
successfully  introduced,  should be promoted. 

10.  The Board  noted  the substantial  strengthening  over  recent years of the interaction between the  Fusion 
Programme  and  industry,  centered  mostly  on ITER  activities.  Industry  will have to play  the  primary 
role  in  the  final  development  of  fusion:  it  is  therefore  recommended  to  increase  progressively  its 
involvement,  particularly  in the System Engineering  area. 

In  addition,  the  following  recommendations  are  made  regarding  other  aspects  of  the  Programme's 
activities: 

11.  The  Board  was  favourably  impressed  by  the  work  performed  on  the  assessment  of  safety  and 

environmental  aspects of fusion  power and strongly encourages a renewed  effort  in this  field. 

12.  A  substantial  effort  should  be  devoted  to  socio-economic  research  on  fusion.  It  needs  to  be  multi-
disciplinary  and  deal  with  issues  such  as:  cost  elements  of  fusion  power,  public  awareness, 
democratic  governance  of complex systems, and value change. 

13.  The  watching  brief  on  Inertial  Confinement  Fusion  should  be  maintained,  and  in  this  frame  a  co 

ordination of the civilian national efforts  in Europe  is recommended. 

14.  The  high  rate  of  human  mobility  achieved  by  the  "Mobility  Scheme"  is  essential  to  the  cohesion  of 
the programme  and  should be maintained.  The domain of application  should be  increasingly  open to 
industry. 

75.  The  decentralised  management  of  the  Programme  is  efficient,  flexible  and  well  accepted  by  the 
scientific,  technological  and  industrial  circles.  With appropriate  adaptations,  it seems  suitable  for  the 
difficult  tasks  ahead.  The  ITER  project  will,  of course,  call  for  a specific,  central  management  of 
the European contributions  when the project  reaches the construction phase. 

And finally,  external events may  invalidate the basic assumptions of this Report : 

16.  A breakdown  of the  ITER  framework  for  any  reason would  result  in the  inability  to go forward  with 
the  planning  and  construction  of  ITER  on  the  lines  broadly  outlined  in  this  report.  In  this  event  the 
Board  recommends  that there should be a complete  reassessment  of the European  Fusion  Programme 
in the light of this new  situation. 

172 

II. 

COMMISSION'S RESPONSE 

1. 

2. 

At  its  meeting  on  26  November  1996,  the  Consultative  Committee  for  the  Fusion  Programme 
(CCFP)  has  unanimously  adopted  the  preliminary1  views,  herewith  attached,  on  the  Board 
Findings  and  Recommendations.  The  Commission,  which  is  a  formal  Member  of  the  CCFP, 
adheres fully  to these views. 

Regarding  the  Board  recommendations  on  ITER,  it  could  be  added  that  the  recent  expression  of 
interest  in hosting  ITER  from  a EU  Member  State  confirms,  as pointed  out by  Mrs  Cresson  at  the 
Research  Council  on  7  October  1996,  the  will  of  Europe  to  play  a  significant  role  in  this 
enterprise.  But  the  extent  of  this  role  cannot  be  defined  before  the  financial  involvement  is  fully 
clarified.  Quadripartite  explorations  have  just  started  among  the  ITER  partners  which  should 
allow,  among  others,  to  draw  up,  in  a  year  or  so,  realistic  assumptions  on  the  sharing  of  the 
financial  burdens between  the potential  host and the other  partners. 

Preliminary  views of the  CCFP on the findings  and recommendations  of the  1996 Fusion  evaluation 
board 

•  The  CCFP  welcomes  the  recognition  by  the  Board  of  the  success  of  R&D  work  performed  and  of  the 
validity  of  the  strategy  followed  by  the  European  Fusion  Programme.  It  accepts  the  need  for  enhancing 
industrial  involvement,  pursuing  actively  safety  and  environmental  studies,  and  launching  studies  on 
socio-economic  aspects. 

•  The  CCFP  understands  that  arguments  in support  of the Findings  and  Recommendations  will  be  found  in 

the full  report. 

•  The CCFP wishes to highlight the following  points from  the Findings and Recommendations  of the Board: 

Fusion  is mature  for  ITER,  and the whole momentum  of the Programme  would  be  lost  if ITER  were 
not built in the near  future. 

Today,  Europe  has  the  leading  role  in  fusion  research.  Now  that  the  construction  of  the  first  fusion 
reactor  approaches,  with  an  increasing  involvement  of  industry,  it  would  be  difficult  to  understand 
that Europe abandons  its leading  role: ITER should be built in Europe. 

Fusion has an important  role to play  in meeting the growing energy  needs of the coming  generations. 
The  CCFP  agrees  with the Board  in pointing  out that  fusion  has to be  seen  in the  context  of  a much 
wider energy  research  effort. 

Following  the  principle  of  subsidiarity,  not  all  energy  options  have  to  be  developed  at  Community 
level, but because of its long-term  and wide scope, fusion  necessarily does. 

•  The CCFP fully  realizes that  important  additional work will have to be performed  as a consequence  of the 

Board  report: 

In preparation  of the  ITER  site decision,  a thorough  description  of  the  long-term  impact  on both  the 
host and the non-host programmes,  respectively,  has to be produced. 

The  future  programme  will  have  to be  re-directed  and  re-focused  after  the  decision  on  ITER  siting 
and  construction. 

'  The  "Findings  and  Recommendations"  of  the  1996  Fusion  evaluation  Board  are  based  on  an  in-depth  analysis,  performed  by  the 
Board, which  will  be  included  in the  full  report yet to be  finalised. 

173 

•  The CCFP agrees with the Board that a negative decision on ITER would have such a large  impact,  that a 

complete re-assessment  of the European Fusion Programme would become  necessary. 

•  The  CCFP  considers  the  Findings  and Recommendations  of the Board  report  to be  an  important  input  for 
the  preparation  of  the  Fifth  Framework  Programme.  When  the  full  report  of  the  Board  will be  available, 
the CCFP will provide  its Opinion. 

174 

/18. 

m. 

EVALUATION  PANEL 

Prof.  Sergio  BARABASCHI 
Former  Under-Secretary  of  State  for  Research  and  Technology,  Italy,  Chairman  of  the  European 
Industrial  Research  Management  Association  (EIRMA) 

Dr Claus BERKE 
Former  Executive  Director  at Siemens  AG and  former  President  of the European  Nuclear  Forum 

Dr Feliciano  FUSTER  JAUME 
Chairman  and  Chief  Executive  Officer  of  Empresa  Nacional  de  Electricidad  (ENDESA)  and  its  Group  of 
companies 

Sir John  HILL 
Fellow  of the Royal  Society,  former  Chairman,  United  Kingdom  Atomic  Energy  Authority 

Prof.  Lars  INGELSTAM 
Professor  of Technology  and  Social Change,  Linkôping  University 

Prof.  Francis TROYON 
Professor  of Physics,  Ecole  Polytechnique  Fédérale  de Lausanne,  Chairman  of the JET  Council 

Prof.  Harry  van der  LAAN 
Professor  of Astronomy,  Univesity  of Utrecht,  former  Director-General,  European  Southern  Observatory 

Prof. Jean-Paul  WATTEAU 
Former  Rector-Chancellor  of Académie de  Grenoble 

175 

19. 

1996 FIVE-YEAR ASSESSMENT 

JOINT RESEARCH CENTER 

OVERALL  EVALUATION  AND SYNTHESIS  OF 

VISITING  GROUPS  REPORTS 

by 

Professor Juan M. Rojo 
Universidad  Complutense de Madrid 

SPAIN 

177 

19. 

L 

EXECUTIVE  SUMMARY 

I have  the  clear  impression  that further  progress  has taken place  in the JRC  since  the  encouraging  analysis 
of  Sir  Hermann  Bondi  three  years  ago.  Both  the  management  of  the  Centre  and  the  Board  of  Governors 
guidance  deserve  much  credit  for  this  steady  evolution. 
It  is  also  worth  remarking  that  most  of  the 
I  am  confident  that 
recommendations  made  in  the  previous  evaluation  have  been  actually  implemented. 
the  recommendations  that  are  included  in  the  excellent  reports  elaborated  by  the  present  Visiting  Groups 
will also be of great benefit  to the Institutes. 

Under  the  guidelines  of  the  Council,  the  JRC  has  obtained  a  much  clearer  definition  of  goals  and  has 
become  much  more  open  and  accessible  to  the  outside,  with  increasing  activities  of  support  to  the 
Commission  services  and  Union  policies  and  widespread  networking  with  research  organisations, 
authorities  and  industries.  In  particular,  the  Centre  has  reacted  quickly  to  the  new  mandate  of 
competitiveness  on  all  fronts:  it  has  successfully  engaged  in  shared  cost  actions  of  the  Framework 
Programme  of  the  Union  and  is  becoming  progressively  more  involved  in joint  ventures  with  industries. 
With  the  refreshment  of  the  management  team,  new  and  intensive  drives  towards  project  control,  quality 
management  and  marketing  are  being  introduced  to  increase  the  visibility  of  the  scientific  and  technical 
activities  and  to  render  the  Centre's  operation  more  transparent  and  understandable.  Moreover,  a  change 
of  culture  is discernible  in which  much  of the staff  now  seems  convinced  of the benefits  accruing  from  the 
more  competitive  stance  of  the  JRC.  Continuous  support  of  this  succesful  policy  is  imperative  and, 
following  the  advice  of  some  of  the  Visiting  Groups,  I recommend  fostering  it even  further  by  setting  up 
on  a progressive  basis  a  user  advisory panel  for  each  Institute,  that  would  promote  relevance  in  project 
missions  and  channel  the  needs  of  the  users  and  their  views  on  the  operation  of  the  Institutes,  particularly 
in relation  to  transfer. 

In order  to accomplish  their  mandated  activities  of  institutional  and  competitive  research  the JRC  has  to be 
an  exceptionally  qualified  partner  and  it  is gratifying  to  note  that  the  JRC  has  succeeded  in  maintaining  a 
number  of core  areas  of scientific  excellence  and  is struggling  to establish  new  ones.  To nurture  the  search 
for  quality, 
I  propose  the  creation  of  a  permanent  scientific  advisory  panel  reporting  to  the  Board  of 
Governors.  This  panel  would  assess  the  quality  of  the  ongoing  research,  with  particular  regard  to 
identifying  high  quality  units  in the  Institutes,  and  would  also  advise  on  exploratory  research.  They  would 
also provide  valuable  inputs to the Visiting  Groups. 

An adversary  of excellence  is uncontrolled  diversification.  Focusing is imperative  as  it seems that  the JRC 
has  much  too  broad  a  spectrum  of  subjects.  It  is  surely  much  better  to  have  a  restricted  number  of  areas 
in  which  the  JRC  is  recognised  as  first-class  than  to  have  at  one's  disposal  a  kind  of  low-profile  general-
purpose  partner.  After  all,  one  must  consider  JRC  activities  with  reference  to  subsidiarity  and  there  is no 
point  in duplicating  national  facilities  if there  is no definite  advantage.  Much  in the  same  line, the JRC  has 
to make  every  effort  to avoid  routine  work,  even if it is requested by third  parties. 

In  order  to  maintain  the  positive  evolution  of  the  JRC,  special  attention  should  be  given  to  the  issues  of 
staff management.  In relation to recruitment,  I have the most serious  reservations  for  the  future.  As noted 
in previous  evaluations  one has to  remark  yet  again  on the  difficulties  in trying  to use  a system  apparently 
designed  to  recruit  government  officials  to  obtain  high  grade  scientists.  It  is  far  too  rigid,  and  much  too 
slow  to  meet  the  demands  of  1996,  let  alone  those  of  the  coming  century.  Some  slight  improvement  has 
been  noted  but  a  substantial  overhaul  is  needed  and  I  suggest  the  urgent  setting  up  of  a  specific  panel  to 
deal  with  the  subject  including  possible  changes  of  current  regulations.  I  have  great  hopes  that  the  recent 
transformation  of  the  JRC  into  an  independent  Directorate  General  might  help  to  implement  the  specific 
requirements  of  a  research  centre  in  the  EC  structure.  Mobility  is  another  key  issue  that  requires 
revitalisation.  The  continuous  flux  of  ideas  to  the  Institutes  should  be  encouraged  by  stimulating  the 
temporary  attachment  of  trainee  scientists  from  other  institutions  and  the  regular  presence  of  research 
students, particularly  at the post-doctoral  level. 

179 

19. 

I  fully  support  the  concerns  of  my  distinguished  predecessor  regarding  the 
A  final  word  about  energy. 
Independently  of  personal  or  political  attitudes  towards  nuclear 
decline  in  nuclear  relevant  research. 
energy  in  Member  States,  nuclear  safety,  waste  treatment,  non-proliferation  and  other  issues,  are  still 
with  us.  They  are  no  respecters  of  national  frontiers  and  deserve  our  attention  at  European  level.  It  is 
worth underlining  that the JRC has first  class expertise  in some areas  of this  field. 

In  summary,  I  believe  that  the  JRC  is  now  on  the  right  track  and  that  present  policies  are  workable  and 
satisfactory  and,  being  supported  by  efficient  management,  have  already  transformed  the  Centre  into  a 
useful  partner  of  institutions  and  industry  in Europe.  In order  to maintain  this  satisfactory  trend  a  number 
of  steps,  for  which  recommendations  are  given,  should  be  taken,  at  least  in  relation  to  staff  management, 
support  of quality,  and  focusing  of  research  subjects. 

1.  Introduction 

Research,  technological  development  and  demonstration  (RTD)  is  one  of  the  key  policies  of  the  present 
Treaty  of  the  Union,  being  third  in  rank  in terms  of  economic  volume.  The  reason  for  this  high  profile  is 
the  conviction  that  those  basic  objectives  of  the  European  Union  (EU),  industrial  competitiveness  and 
improvement  of  the  quality  of  life,  can  only  be  built  on  the  solid  foundations  of  science  and  technology. 
Whereas  the  commitment  of  the  EU  to  RTD  has  been  permanent  and  increasing,  the  emphasis  on  the 
different  areas,  or  even  in the  type  of  activities  required,  has been  continuously  changing.  The  role  of  the 
Joint  Research  Centre  (JRC) has also undergone  a number  of  realignments. 

Following  the  dwindling  interest  in  fission  research  in  the  past,  specialised  nuclear  establishments  in 
Member  States  had  to  face  arduous  reconversions.  The  JRC  was  no  exception  and,  in  fact,  some  of  its 
present  specialisations,  like  environment  or  safety,  coincide  with  the  goals  of  some  of  the  reconverted 
national  nuclear  centres.  On  these  grounds  it  is not unexpected  that  in the past  the JRC  strategy  was  often 
defensive,  nor  is  it  surprising  that  in the  last  few  years,  there  have  been  a  number  of  definitions  from  the 
European  decision-making  bodies  in  relation  to  the  goals  to  be  attained  by  the  Centre  and  to  the  various 
procedures  of  evaluation  and  assessment  of  its  performance.  The  last  of  them,  in  1994,  is  the  so-called 
'modified  role'  of the JRC  as agreed  by the Council,  Parliament  and  Commission.  Among  other  issues this 
new  role  emphasised  the  need  for  the  JRC  to  be  more  involved  in the  denominated  'competitive  actions', 
often  in partnership  with  enterprises. 

Although  the  present  assessment  is  supposed  to  cover  the  period  of  five  years  1992-1996  (with  full 
recognition  of  the  previous  evaluation  for  the  1992-1994  period,  the  Bondi  report),  the  introduction  at 
mid-term  of  the  modified  role  of  the  JRC  referred  to  above  suggests  that  stress  should  be  laid  on  the 
performance  of  the  Centre  in terms  of this modified role,  i.e.  during  the  last two years.  This  has been  well 
understood  by  the  Visiting^ Groups  to  the  individual  JRC  institutes  which,  in  their  reports,  have  given 
special  attention  to the  implementation  of the new  more  competitive activities.  These  reports underline  the 
positive  evolution  of  the  JRC  performance  over  the  last  two  years,  confirming  the  earlier  expectations  of 
the Bondi  assessment. 

The  significance  of  the  individual  reports  of  the  Visiting  Groups  can  hardly  be  over-emphasised  as  they 
have  been  conducted  in  the  most  competent  and  professional  way.  Apart  from  a  limited  input  derived 
from  personal  meetings  with  Institute  Directors  and  staff,  most  of the present  assessment  is based  on  those 
reports.  The  scheme  of  my  own  report  is  as  follows: 
In  Section  2,  I  underline  key  points  from  the 
individual  reports,  trying  to  look  for  unifying  issues.  I then  proceed  to  a  more  general  discussion  of  the 
achievement  of  the  JRC  in  the  last  five  years  (Section  3),  keeping  in  mind  the  objectives  defined  in  the 
corresponding  decisions  of the  European bodies.  In Section 4,  on the basis of the positive  evolution  of  the 
Centre,  I  look  into  the  future  in  search  of  those  R&D  problems  which  the  JRC  can  actually  undertake 
Finally,  Section  5  deals  with  conclusions  and 
more  efficiently 
recommendations,  the  latter  being  general  in  scope.  They  are  to  be  considered  as  complementary  to  the 
specific  ones  suggested  for  the  individual  Institutes by the Visiting  Groups. 

than  national 

laboratories. 

180 

2.  Analytical  remarks on the Individual  Institutes 

19. 

The full  reports of the individual Visiting Groups are published separately.  They encompass an evaluation 
of  the  activities  of  the  Institutes,  in  the  light  of  the  objectives  defined  by  the  Council  decisions,  and 
specific  recommendations  for  the  future.  In the present  Section  2,  I attempt  to  select  from  those  reports 
some guidelines that I find most illuminating or representative of current issues. 

Institute for Reference Materials and Measurements (IRMM): 

* The Institute  appears  to occupy  an important place  in the  EU structure  of  the  sector.  Indeed,  it seems 
that if the Institute did not exist, a centre of this type would have to be created. 

* The Institute seems to have attained  a satisfactory  degree of competence  and  respectability,  particularly 
in nuclear reference  and measurements.  It also embodies special experimental  facilities. 

*  Accountancy  and  management  procedures  require  improvement  and  adaptation  to  the  Institute's 
particular role. 

* Original  research  work must not be neglected.  In this respect  10% of the resources  seems a judicious 
figure  for exploratory research. 

Institute for Transuranium Elements (ITE) : 

* In my view, this is the most qualified  Institute in the JRC. Every effort  should be made to maintain this 
status, particularly  in recruitment issues in the face of coming retirements. 

*  The Institute  has  also been  successful  in  incorporating  competitive  activities  into  their  former,  mostly 
research oriented, ones. 

*  Avoid  remaining  in  areas  which  become  routine:  rather  transfer  work  to  other  establishments  when 
routine approaches.  One should make sure that room for stimulation of creativity exists in all areas (i.e. in 
safeguards analysis). 

*  In  the  two  directions  of  exploratory  research  some  novel  ideas  have  originated  recently  in  other 
European  laboratories  (i.e. the  'energy  amplifier  and waste burner').  ITE should  evaluate them,  report to 
the Commission and, if convenient, network. 

Institute for Advanced Materials (IAM) : 

* Very much restructured,  the Institute has gained  recognition  in the field  of high temperature  materials, 
although  there  is  still  room  for  improvement  in  other  fields.  Diversification  into  already  well-covered 
fields should be avoided. 

* The future  of the tritium handling  laboratory  (ETHEL) should be examined  in view of the evolution of 
the  ITER  and  related  fusion  programmes.  The  relevance  of  this  facility,  in  relation  to  existing  ones  in 
Member States, should be clarified. 

*  Concern  is  expressed  in  a  decrease  in  researchers  being  exchanged  with  external  well-known 
institutions, compared to an earlier situation. This seems to me a critical issue. 

181 

*  Transfer  to  the  outside  seems  to  be  improving  although  research  at  the  Institute  should  better  remain 
generic  and  near,  but  upstream,  of  industrial  research.  The  Visiting  Group  considers  that  shared  cost 
actions  should  be  favoured  and  encouraged. 

19. 

Institute for  Systems,  Informatics  and Safety  (ISIS)  : 

* This  Institute  has been  formed  from  the merging  of two former  ones,  a new  Director  has been  appointed 
and  is, naturally,  still  defining  goals and  procedures.  The management  showed  good perception  as well  as 
determination  and  should  reap  future  rewards. 

*  In  recent  times, 
the  Institute  has  made  a  great  effort  in  working  together  with  external  research  and 
industrial  bodies  and  this  competitive  attitude  has to be  stimulated  and  supported.  The proposed  exchanges 
of  personnel  with  Directorates  General  in  Brussels  appear  as  a  good  practice  for  reinforcing  a  users 
approach  with  these  units. 

*  Quality  seems  to  be  generally  satisfactory.  Perhaps  the  very  wide  spectrum  of  interests  of  the  Institute 
could  be  modulated  by  actual  capabilities  at  the  highest  level.  In  some  areas,  improving  scientific 
leadership  seems  desirable. 

*  The  Institute  includes  some  unique  facilities  whose  maintenance  at  the  highest  level  of  performance  is 
mandatory.  Enhanced  use of these  facilities  by outside users  is  recognised. 

Environment  Institute  (EI)  : 

*  The  Institute  seems  to  be  on  the  right  track  to  the  fulfilment  of  both  institutional  and  competitive 
activities  with  a  fair  level  of  quality.  The  willingness  of  the  management  to  convert  it  into  'the  leading 
institute  in  Europe'  with  'a  high  degree  of  visibility'  has  to  be  made  compatible  with  increasing  activities 
at national  level  in these  areas  and  certainly  requires  careful  focusing  of the actions. 

*  Along  with  a  proper  management  control  the  management  seems  to  be  successful  in  stimulating  the 
generation  of  new  ideas,  without  which  institutional  and  competitive  activities  could  not  be  carried  out 
efficiently. 

*  Institutional  activities  are  crucial  in  this  domain.  For  example,  the  EI  has  produced  amazing  results 
from  the  comparison  of  pollution  measurements  in  blind  tests  performed  by  different  national  reference 
laboratories  with  a  scatter  of  more  than  50%.  It  is  hard  to  overemphasise  the  incidence  of  this  work  on 
implementation  of environmental  policies. 

*  I  strongly  support  the  Visiting  Group's  suggestion  of  developing  to  capacity  the  activities  in  water 
research  and  management  of  water  resources.  An  independent  and  neutral  reference,  along  with  the 
required  scientific  quality,  is mandatory  in this  field. 

Space Applications  Institute  (SAI): 

*  Institute  with  unique  experience  on  télédétection  and  earth  observation.  It  would  benefit  from  stricter 
control  of  scientific  quality  (stimulate  publications  in prestigious journals)  and  performance  in  competitive 
projects  (better definition  of specific  goals and milestones). 

*  As  remote  sensing  is  generic  by  nature  and  aids  in  resolving  so  many  different  problems,  one  has  to 
restrain  the  tendency  of  the  Institute  to  become  specialised  in  too  wide  a  range  of  subjects.  It  would  be 
better  to restrict  activities  to the Institute's  core competences  (remote  sensing  and  application  of  networks). 

*  Interdisciplinarity  requires  improved  co-operation  with  other  Institutes,  particularly  the  Environment 
Institute. 

182 

19. 

Institute for  Prospective  Technological  Studies (IPTS): 

* Practically  new-born,  the Institute  is starting  to establish  a sound  reputation  with  its customers  and  set up 
infrastructure  to  meet  their  future  demands.  The  location  needs  communications  and  world-wide-web  to 
be developed  to full  strength.  The Institute  has still to demonstrate  high quality  and  gain  full  recognition. 

*  Previous  recommendations  on  recruitment  for  Seville  and  launching  of  the  European  Science  and 
Technology  Observatory  (ESTO) have been  accomplished. 

*  The  general  recommendations  on  'focusing'  are  still  more  meaningful  for  IPTS.  Work  should  not  be 
encouraged  in areas where  European  industries  and  laboratories  have  a long established  tradition. 

*  The  Institute  is taking  advantage  of  its geographical  position  for  developing  leadership  in  Mediterranean 
and  North  African  problems.  I  firmly  believe  that  this  approach  should  be  reinforced,  perhaps,  as  the 
Visiting  Group  indicated,  by  developing  a  programme  outside  the  TEC  (Technology,  Employment, 
Competitiveness). 

3.  General assessment of the JRC achievement 

In  his  report  for  the  period  1992-1994,  Sir  Hermann  Bondi  stated  that  'the  JRC  is  well  on  the  way  to 
becoming  a  recognised  centre  of  excellence  in many  of  its areas  of  activity'.  It  is a pleasure  to  be  able  to 
confirm  steady  progress  towards  this  desirable  goal  two  years  later.  By  taking  together  the  five  year 
period,  from  1992  till  the  present,  one  can  safely  say  that  the  JRC  has  successfully  found  a  specific 
position  in the  European  RTD system  and that  it is performing  its role with  increasing  capabilities. 

Under  the  guidelines  of  the  Council  the  JRC  has  attained  a  much  clearer  definition  of  goals  and  has 
become  much  more  open  and  accessible  to  the  outside,  with  increasing  activities  of  support  to  the 
Commission  services  and  Union  policies,  and  in  widespread  networking  with  research  organisations, 
authorities  and  industries.  With  the  refreshment  of  the  management  team,  new  and  intensive  drives 
towards  project  control,  quality  management  and  marketing  are  being  introduced  to  increase  the  visibility 
of  the  scientific  and  technical  activities  and  to  render  the  Centre's  operation  more  transparent  and 
understandable  Moreover,  a  change  of  culture  is  discernible  in  which  much  of  the  staff  now  seem 
convinced  of  the  benefits  accruing  from  the  more  competitive stance  of  the JRC.  It  is perhaps  too  early  to 
assess  the  impact  of  new  distribution  of  tasks  among  the  Institutes,  as  some  merging  and  reorientation  has 
recently  taken  place,  but  it  is  clear  that  further  strengthenening  the  coordination  among  the  Institutes  is 
desirable. 

It  is  to  be  remarked  that  business-like  management  can  never  be  at  odds  with  scientific  and  technological 
quality.  One  cannot  speak  of  transfer  if  there  are  neither  significant  ideas  or  know-how  to be  transferred. 
An  unreasonable  fixation  on  demonstrating  'usefulness'  should  not  obscure  the  fact  that  in  order  to  gain 
competitiveness,  recognition  and  selected  support  of  high  quality  units  in the  Institutes  should  be  assigned 
a  high  priority.  In  fact,  the  JRC  has  succeeded  in  maintaining  a  number  of  core  areas  of  scientific 
excellence  and  is  struggling  to  establish  new  ones,  although  much  thought  has  to  be  given  to  recruitment 
policy  if one  wants  to maintain  that promising  trend.  Also,  the JRC  seems  to have been  evolving  into  less, 
bureaucratic  and  more  transparent  managerial  procedures.  With  regard  to  both  items,  it  is  satisfactory 
that  the  JRC  has  recently  become  an  independent  Directorate  General  as  it  is  hard  to  use  a  system 
apparently  designed  to manage  administrative  offices  to run  an RTD  centre. 

Whereas  the  present  situation  of  the  JRC  can  be  regarded  with  confidence,  there  are  a  number  of  issues 
that  are  still  a matter  of  concern.  I refer  to the most  important  ones perceived  by  the  Visiting  Groups,  and 
in full  agreement  with my own  experience: 

1.  Personnel.  A  matter  of  common  concern  as  this  is  certainly  the  key  point  for  the  future  of  the  JRC. 
One  should  make  sure  that  the  massive  retirement  that  is expected  in  the  coming  years  is  offset  by  hiring 
younger  people  of  the  highest  quality.  Recruitment  procedures  are  still  far  from  efficient.  The  whole 

183 

19. 

process of recruitment should be, tuned so that 3 years contracts could be used as a test of JttD  capabilities 
for  future  candidates to temporary (semi-permanent) positions.  I believe that only an overall evaluation of 
the problem can lead to substantial improvement and I specifically  advise setting up a special panel for this 
purpose.  As is often  stated that much needed  reforms  are contrary to EU rules, maybe changing some of 
these rules for RTD centres could be one of the recommendations of that panel. 

2. Exchange of researchers. An action much praised  by some of the Institutes in the past,  particularly in 
strengthening capabilities and training young researchers in emerging areas.  It also facilitates  networking. 
However,  it seems to be receding and correction of this trend should be undertaken. 

3. Focusing.  There seems to be much too broad a spectrum of subjects.  It is surely much better to have 
a restricted  number of areas  in which the JRC is recognised  as first  class than to have at one's disposal a 
kind of low-profile,  general-purpose partner. 

4.  Management and Accountancy.  Some advance since the last report is recognised  (i.e. rapid response to 
manpower  needs  in  third  party  projects).  However,  improvements  are  still  widely  requested.  An 
appraisal of general accountancy procedures is suggested. 

5.  Co-ordination.  Further  advances since the last report are recognised,  but stronger co-ordination among 
the  Institutes  is  required.  The  Directorate  of  Programmes  seems  to be  working  at  a very  reduced  staff 
level and requires examination. 

6.  Nuclear  energy.  Apart  from  its  historical  origins,  the  JRC  is  the  depository  of  much  European 
knowledge  in this  subject,  perhaps  the only  one  in which  it  is recognised  as the true  European  leader in 
some areas.  In no way should this capital asset be wasted. 

4.  A look  into the  future 

The JRC has received  the mandate of performing  three types of activity:  institutional,  competitive  inside 
the  Framework  Programme  (FP)  and  competitive  outside  the  FP.  Positive  response  in the period  under 
examination to this challenge suggests that the time may be ripe for  a further  step, which would be that of 
progressively  concentrating  JRC  actions  on  those  in  which  its  European added  value (EAV)  is  fully 
exploited.  In other  terms  it  would  mean proceeding  from  the  question:  Is there  anything  useful  that we 
can  do  with  the  infrastructure  and  personnel  in  the JRC?  to  the  much  more  engaging  one:  Are  there 
R&D  areas  or  activities  in which  the  existence  of  a European-owned  research  laboratory  is desirable  or 
even necessary?  Indeed,  the overall RTD policies of the European Union (EU) are built on the principle 
of subsidiarity.  As is well known, the EU Member States have a number of excellent laboratories, both at 
their  universities  or  national  institutes  and  at  their  industries.  One  may  wonder  whether  there  is  in 
principle scope for  an institute at European level, in which targeted research  is performed.  If the  functions 
of  such  a centre  can be  recognised,  one  can  further  analyse  if  the JRC  is presently  capable  of  fulfilling 
these requirements. 

On the basis of the  Council  Conclusions  of 26 April,  19941one could  envisage three possible  domains in 
which the JRC would have full EAV: 

The first domain would be that of specific  and difficult  problems that  required,  for  their  solution,  critical 
mass of  scientific  and  technical  excellence.  Excellence  would  be promoted  by  drawing  from  the  pool of 
bright people in the different  Member  States and integrating that talent in first-class  units of critical mass, 
that  would  furthermore  pervade through the European RTD system by means of  suitable  networking  and 
partnership.  The  second  domain  would  be  that  of  developing,  maintaining  and  managing  large 
experimental facilities, which could not be run independently by Member States.  In the past a number of 
large  experimental  facilities  have  been  set  up,  or  upgraded,  in  Europe  on  the  basis  of  transnational  co 
operation  among  different  countries.  The  third  domain  would  be  that  of  carrying  out  research  and 

OJNo. C126, 7.5.1994,  p.l 

184 

development  actions  in  which  impartiality  and  neutrality  were  mandatory.  Needless  to  say,  these  virtues 
would have no practical use if they were not accompanied by scientific  quality  and  recognition. 

Let  us  now  turn  to  the  crucial  question,  that  is  if  one  can  realistically  consider  that  the  JRC  can  be 
competent  in all these three domains at the present time: 

19. 

• 

• 

• 

First,  its  possible  role  as  hosting  units  of  excellence  is  analysed.  Needless  to  say,  I  refer  to  both 
scientific  and  technical  excellence.  Gaining  excellence  is not  at  all  an easy  task:  if  an  institute  has 
little  prestige,  it  will  hardly  attract  the  brightest  people,  which  in  turn  will  result  in  further 
stagnation.  One has to recognise  that the JRC has rarely  become  a reference  point  in the past,  but 
the  situation  is  steadily  improving  as  discussed  in  Section  3.  The  present  management  has 
succeeded  in  injecting  morale  and  determination  into  the  staff,  which  now  seems  to  believe  in  its 
own capabilities.  Nevertheless,  some bureaucratic  remnants  are  still noticeable;  for  example,  some 
officers  when  describing  their  work  appear  to be  seized  by  the how  but  reluctant  to  emphasise  the 
what.  Full  support  of emerging  new and promising  ideas should be prevalent  in all the Institutes. 

As  support  must  always  be  selective,  the  management  of  the  Institutes  would  much  benefit  from 
accurate  information  concerning  the  external  appraisal  of  the  quality  of  the  different  groups.  I 
believe  that  the  Board  of  Governors  (and  the Visiting  Groups  for  their  periodic  assessment)  would 
very  much  benefit  from  the  creation  of  a scientific  advisory panel  that  assessed  the  quality  of  the 
ongoing  research,  with  particular  regard  to  identifying  high  quality  units  in  the  Institutes,  and 
advised  also  on exploratory  research.  One  must  not  forget  that  only  high  quality  units  can  actually 
provide  EAV to their competitive partnerships with European  industries  and  institutions. 

In  the  pursuit  of  excellence,  there  are  a  number  of  issues  in  which  improvement  is  desirable. 
Ensuring  that  the  best  young  people  apply  and  that  selection  is  efficient  is  second  to  none  among 
the elements to maintaining  high  standards.  My  advice  about personnel  reforms  is here  crucial,  as 
is  my  recommendation  about  focusing.  A  further  suggestion  in  relation  to  mobility  also  seems 
relevant.  It  is  always  difficult  to  attract  on  a permanent  basis  first-class  scientists  or  technologists 
who  are  well  established  in  other  institutions.  However,  incorporating  them  into  the  JRC  groups 
for  limited  (but not too  short)  periods  could  result  in  immense  benefit.  Perhaps,  previous  practices 
of  the  Institutes  to  incorporate  external  researchers  could  be  reinforced  by  developing  a  special 
programme  which  took  advantage  of  sabbatical  leave  and  included  some  convenient  incentives  for 
attracting  these highly  rated  researchers. 

Let  me  now  turn  to the second  function,  that  of  managing  large facilities,  which  can  become  also 
reference  points  for  networking  with  other  European  institutions.  It  is worth  remarking  that  none 
of  the  recently  developed  large  facilities  in  Europe  has  been  undertaken  under  the  JRC  umbrella. 
Even  the  first  European  tokamak  (JET),  although  funded  by  the  Framework  Programme  of  the 
EU,  was  conceived  and  developed  outside  the  JRC.  The  positive  evolution  of  the  JRC, 
continuously  alluded  across  this  report,  might  change  the  attitudes  of  the  Member  States  in  the 
future  and  some  thought  should  be  given  to  alleviate  some  of  the  pending  obstacles.  The  new 
Articles  130 k,  1 and  n of the Union Treaty  might also help to implement this  function. 

Finally,  I  address  the  function  of  impartiality  and  neutrality.  It  is  often  stated  that  Union  policies 
require  support  from  the  JRC  but  very  often  this  support  is  thought  of  in  terms  of  routine  or 
bureaucratic  work.  This  is certainly  wrong  and  creative  and  original  work  is mandatory  for  these 
purposes.  This  is  perhaps  the  role  in  which  the  JRC  has  made  a  most  visible  progress  recently. 
The  support  of  this  role  should  continue,  but  one  must  not  forget  that  this  support  would  become 
meaningless  if  the  quality  of  JRC  research  failed  and,  consequently,  a  subtle  equilibrium  between 
the  first  and third domains  is imperative. 

Last,  but  not  least,  the  development  of  a  body  of  independent  knowledge  in  scientific  and 
technological prospection  is crucial  for  complementing  the  inputs from  the different  Member  States 
Previous  Framework 
in  order  to  formulate  the  basis  of  future  Framework  Programmes. 

185 

Programmes  have  suffered  from  the  lack  of  a  neutral  and  recognised  scientific  and  technological 
prospection  and  some  of  their  programmes  have  become  a  mere  list  of  Member  States'  interests, 
often  without  much  EAV.  The  JRC  has  still  to  gain  prestige  and  full  recognition  in  this  task  but 
the  rewards  can be  vast. 

19. 

5.  Conclusions  and  Recommendations 

I  have  the  clear  impression  that  considerable  further  progress  has  taken  place  in  the  JRC  since  the 
encouraging  analysis  of  Sir  Hermann  Bondi.  Both  the  management  of  the  Centre  and  the  Board  of 
Governors  guidance  deserves  much  credit  for  this  steady  evolution.  In  particular,  it  is  worth  remarking 
that most of the recommendations  made  in the previous  evaluation have been  actually  implemented. 

The  mandate  to  enter  the  competitive  arena,  and  the  general  positive  answer  to  this  challenge  in  the 
different  Institutes,  has had  a number  of favourable  consequences: 

0  Providing  a check of quality  (competing  with national teams, having to attract potential users  etc.) 
0  Opening  the JRC  groups to the outside,  i.e.  networking,  etc. 
0  Improving  management  procedures, 

like  quality  management,  project  management  and  active 

marketing. 

The  first  recommendation  is,  consequently,  that  of  maintaining  and  supporting  the present  policies.  One 
has  to realise,  however,  that  even  for  consolidating  the present  advances,  a number  of  improvements  have 
to  be  made,  particularly  in  the  area  of  personnel  management.  Also,  looking  into  the  future,  one  has  to 
aim  at  establishing  the  JRC  as  an  institution  with  full  EAV  in the  European  RTD  community  and  this  will 
require  further  adjustments.  In the  following,  I summarize  a number  of  recommendations  that  derive  from 
the  analysis  of the previous  sections: 

a)  Recommendations  for  specific  action. 
•  Set  up  urgently  a  special  panel  of  experts,  including  specialists  in  European  legislation,  to 
advise  on  the  subject  of  personnel  status  and  recruitment  procedures,  including  possible 
changes  of current  regulations. 

•  Explore  the  possibilities  of  developing  a special  programme  for  the  incorporation  of  first-class 
scientists  into  JRC  laboratories  during  their  sabbatical  leave,  that  would  reinforce  previous 
individual  action  by  the  Institutes.  Ensure  a  continuous  incorporation  of  research  students, 
particularly  at post-doctorate  level to the Institutes. 

•  Set  up  an  external  scientific  advisory panel,  reporting  to  the  Board  of  Governors,  that  would 
assess  the  quality  of  the  ongoing  research,  with  particular  regard  to  identifying  high  quality 
units  in the  Institutes,  and  advise  also  on  exploratory  research.  The  panel  would  also  help  with 
the  selection  of new  scientific  staff  and  exchanges. 

•  Set  up  user  advisory  panels  at  the  Institutes  in  which  the  convenience  of  their  existence  has 
been manifested  (for  example,  ISIS) and,  on the basis of ongoing  experience,  extend  them to all 
the  Institutes. 

b)  Recommendations  for  general  policy 
•  Avoid  excessive  proliferation  of  subjects  in  the  Institutes.  Instead,  try  focusing  on  activities 

leading to attainment  of first-rate  status  in a limited number  of  subjects. 

•  Emphasise,  in  the  current  accounts  of  the  JRC  activities,  concrete  results  obtained  in  JRC 
Institutes  rather  than  organisational  issues.  Periodically  publish  and  disseminate  news  about 
those  achievements.  If  possible  select  those  cases  in  which  the  JRC  has  been  given  significant 
recognition  (as opposed  to  'listing  customers'). 

186 

•  Stimulate  an ambience  in which original,  promising  ideas can arise  and  support  them  in order 
to  explore  their  possibilities  Encourage  publication  in  prestigious  journals  in  order  to  gain 
recognition.  Ensure that sufficient  effort  is devoted to exploratory  research. 

•  Avoid  routine  work  even if it is sought by third parties.  Try  to step out  of  certain  lines, and 
leave their continuation to other centres once the EAV of the action has vanished. Nevertheless, 
be careful  in this respect in those areas in which the JRC has unique facilities. 

•  Undertake  the  necessary  steps  to  support  the  JRC  as  a  candidate  for  future  European  large 

facilities.  Explore the possibilities of using Articles  130 k, 1 and n as a 'driving  force'. 

A note on the evaluation  process  itself 

In general,  the  evaluation  process  itself  seems to fulfil  its original  purpose.  In  fact,  previous  evaluations 
have  actually  contributed  to marked  improvements  in different  aspects  of JRC  management.  There  are, 
nevertheless, aspects that can be ameliorated: 

•  The  Visiting  Groups'  task  could  be  supported  by  an external  evaluation  of  the  quality  of  the 
output  of  the  Institutes  and  by  a unified  view  of  the  users  appreciation  of  transfer,  competitive 
partnership and related matters. The proposed panels could well fill this gap. 

•  The first  point of the terms of reference  of the Visiting Groups states that the evaluation  refers 
to  all  the  objectives  of  the  Council  Decisions  of  15  December,  1994  .  However,  the  Visiting 
Groups'  reports  appear  rather  unbalanced  in relation  to the different  objectives;  for  example,  the 
sixth objective  of the JRC  "contributing  to the reduction  of scientific  and technological  disparities 
between  Member  States"  does  not  seem  to  have  played  an  easily 
recognisable  role  in  the 
evaluation. 

Acknowledgements 

I acknowledge  very useful  discussions  with the Board  of Governors  and members  of the  Management of 
the Centre and of the Visiting Groups.  The exchange of views on the report  with Sir  Hermann Bondi has 
been of immense benefit  to me. 

OJ No. L361, 31.12.94, p.l 14 and p.132 

187 

11^  MEMBERS  OF THE VISITING  GROUPS 

A. 

Institute  for  Reference  Materials  and  Measurements  (IRMM)  -  Geel 

19. 

Dr.  Maria  Grazia DEL MONTE  (Chairwoman) 
Director 
National  Centre  for  Reference  Materials 
ITALY  - ROME 

Mr.  Peter J.  JENKS 
Product  Manager 
Biological,  Elemental  and 
Pharmaceutical  Reference  Materials 
Promochem  Gmbh 
GERMANY  - WESEL 

Mr.  Robert  KAARLS 
Director 
Nederlands  Meetinstituut  (NMi) 
THE  NETHERLANDS  -  DELFT 

B. 

Institute  for  Transuranium  Elements  (ITE)  -  Karlsruhe 

Dr.  Niels HANSEN(Chairman) 
Head 
Materials  Department 
Riso National  Laboratory 
DENMARK  - ROSKILDE 

Mr.  Richard  HOOPER 
Director 
IAEA 
AUSTRIA  - VIENNA 

Dr.  Klaus  JANBERG 
Geschaftsfuhrer 
GNS 
Gesellschaft  fur  Nuklear-Service  GmbH 
GERMANY  - ESSEN 

Mr. Jean van VLIET 
Manager 
MOX  plant 
Belgonucléaire 
BELGIUM  - DESSEL 

C. 

Institute  for  Advanced  Materials  (IAM)  - Petten  and  Ispra 

Prof.  Claude BATHIAS  (Chairman) 
Conservatoire  National  des  Arts 
et Métiers  (CNAM) 
FRANCE  - PARIS 

188 

19. 

Prof. Pier Giulio AVANZINI 
Direzione Area della Ricerca 
Consiglio Nazionale Ricerche 
ITALY - GENOA 

Mr. Staffan SÔDERBERG 
Director 
AB Sandvik Coromant 
SWEDEN - STOCKHOLM 

Prof. Dr. T. VARGA 
Leiter der Techn. Versuchsund  Forschungsanstalt 
TU Wien 
AUSTRIA - WIEN 

D. 

Institute for  Systems,  Informatics  and Safety  (ISIS) - Ispra 

Mr. Peter BENTON (Chairman) 
Director General 
British Institute of Management 
UNITED KINGDOM - LONDON 

Prof. Jane GRIMSON 
Dean of Engineering and Systems Sciences 
Department of Computer Science 
Trinity College 
IRELAND - DUBLIN 

Mr. Theodossios P. TASSIOS 
National Technical University of Athens 
GREECE - ATHENS 

Prof. Dr. Dennis TSICHRITZIS 
Chairman of the Executive Board 
GMD-Forschungszentrum 
Informationstechnik  GmbH 
GERNANY - SANKT AUGUSTIN 

Dr. F. YNDURAIN 
Director General 
CIEMAT 
SPAIN - MADRID 

E. 

Environment  Institute (EI) - Ispra 

Prof. Carlos A. BORREGO (Chairman) 
Department of Environment and Planning 
University of Aveiro 
PORTUGAL - AVEIRO 

Prof. A.R. HARDY 
Research Director 
Central Science Laboratory 
UNITED KINGDOM - SLOUGH BERKS 

189 

19. 

Prof.  Dr.  A.  KETTRUP 
Director 
GSF  Neuherberg 
Institute  of Ecological  Chemistry 
GERMANY  -  NEUHERBERG 

F. 

Space  Applications  Institute  (SAI)  -  Ispra 

Prof.  Seppo URPO  (Chairman) 
Director 
Metsâhovi Radio Research  Station 
FINLAND  - KYLMALA 

Prof.  Dr.  Herbert  FISCHER 
Head 
Institut  fur  Météorologie und  Klimaforschung 
Forschungszentrum  Karlsruhe 
GERMANY  -  KARLSRUHE 

Mr.  Roy  GIBSON 
Former  Director  General  of ESA 
FRANCE  -  MONTPELLIER 

G. 

Institute  for  Prospective  Technological  Studies  (IPTS)  -  Seville 

Dr.  Robert  C.  WHELAN  (Chairman) 
Former  Chief  Executive  of  CEST 
UNITED  KINGDOM  -  CAMBRIDGE 

Dr. Josef  FRÔHLICH 
Head,  Division  of  Systems  Research 
Austrian  Research  Centre  Seibersdorf 
Technology-Economy-Environment 
AUSTRIA  -  SEIBERSDORF 

Prof. Dr.  Herbert  PASCHEN 
Leiter  des Instituts  fur  Angewandte  Systemanalyse 
und  Technikfolgenabschàtzung  (ITAS) 
GERMANY  -  KARLSRUHE 

Prof. Walter  ZEGVELD 
Chairman 
Rathenau  Institute 
THE NETHERLANDS  - THE  HAGUE 

190 

Annex 

191 

ANNEX 1 

TERMS OF REFERENCE 
FOR THE 5-YEAR ASSESSMENT OF THE 
SPECIFIC PROGRAMMES 

1. 

Assessing relevance of programme objectives : 

Are the initial objectives still valid against evolving S&T, industrial and socio 
economic conditions ? 
Have the selection criteria as outlined in the Framework Programme Decisions 
been adhered to ? 

2. 

Assessing efficiency  : 

Has the programme sought to achieve/achieved its objectives in a cost  effective 
manner ? 
How efficient  has the overall management of the programme been ? 
Have objectives been adopted against which performance can be evaluated ? 
Is improvement to the external monitoring process/procedure necessary ? 

3. 

Assessing effectiveness  : 

Have the initial  objectives of the programme been achieved, or, for the longer 
term strategy and objectives,  is progress towards achieving them sufficient  ? 
Have the results stemming from the programme been disseminated/exploited ? 
Have the overall objectives of the Framework Programme been achieved, 
particularly : 
•  What have been the contributions to strengthening the scientific and 

technological bases ? 

•  What has been the interaction with and/or what have been the contributions 

to relevant Community policies such as cohesion, competitiveness, agriculture, 
employment, cooperation for development, economic cooperation, fisheries,  etc. ? 

•  Is Community RTD activity sufficiently  coordinated with that of the Member 

States ? 

How is European "added value "/cost benefit/subsidiarily  best demonstrated ? 
Have the results stemming from the programme been disseminated/exploited ? 

4. 

Examples of major  achievements : 

Outstanding S&T results and innovations. 
Major  achievements as regard the contributions to other community policies. 

5. 

Lessons learned from programme implementation : 

What lessons have been learned from programme implementation ? 
Success stories of collaboration. 
Failure stories of collaboration. 

6. 

Recommendations for the next programme. 
The five-year assessment will pay particular attention to the coherence between 
Community andnational S&T policies with a view to enhancing their mutual consistency. 

192 

ANNEX 2 

TERMS OF REFERENCE 
FOR THE VISITING GROUPS FOR THE EVALUATION OF THE 
JOINT RESEARCH CENTRE'S INSTITUTES 

1.  To evaluate progress in performing  work in accordance with the objectives set out in Council Decisions 
94/918/EC  and  94/919/Euratom,  documents  addressed  to the  Board  of  Governors,  the JRC's  Annual 
Work  schedules,  budget  etc.,  taking  into  account  the  results  of  the  last JRC  evaluation  performed  in 
1993-94. 

2.  To  ensure  that  the  Institute  has  effective  mechanisms  to  set  its  local  scientific  objectives,  monitor 

progress towards those objectives and evaluate outputs. 

3.  To review the various research activities to assess: 

a)  the  relevance  of the work  with  respect  to Community  needs,  including  the  relevance  of  work 
performed  for  external  customers,  when  applicable.  In  particular  whether  the  objectives 
initially set for the Institute are still valid against evolving  S&T,  industrial and  socio-economic 
conditions; 

b)  the scientific quality and productivity of the Institute; 

c)  whether full advantage is being taken of scientific opportunities in the light of funding  and other 

constraints. 

4.  To assess the effectiveness  of the scientific management leadership within the Institute. 

5.  To advice on whether the Institute has been making efficient,  effective  and economical use of resources 
in  carrying  out  its  programmes  and  management  functions.  Resources  include  manpower,  money, 
services, facilities, data and equipment. 

6.  To review the extent and effectiveness  of the Institute's  external  scientific  links,  including  cooperation 
with  research  organisations,  the  higher  education  sector,  industry  and  government  departments  in the 
Member States. 

7.  To  examine  the  relationship  of  the  Institute's  work  in  general,  to  the  mission  of  the  Joint  Research 
Centre  and  to  its  forward  strategic  planning  as  well  as  to  the  overall  objectives  of  the  Framework 
Programme.  Therefore  assessing : 

• 

the contribution towards the improvement  of the competitiveness  of European  industry  and the 
economic impact (both direct and indirect) that has already been plus that which is expected to 
be achieved; 

•  The contributions to strengthening the scientific and technological bases; 

•  The  contributions  to relevant  Community  policies  such  as environment,  industry,  agriculture, 

transport, employment, cohesion, etc. 

To assess the European added value of the Institute's activities on the basis of the subsidiarity 
principle. 

8.  To make recommendations and report to the Board of Governors of the Joint Research Centre. 

193 

ISSN 0254-1475 

COM(97) 149  final 

DOCUMENTS 

EN 

12  15  16  05 

Catalogue number  :  CB-CO-97-140-EN-C 

ISBN 92-78-18077-7 

Office  for Official  Publications of the European Communities 

L-2985 Luxembourg