Source: EURLEX
Language: en
Format: md

**COMMISSION OF THE EUROPEAN COMMUNITIES**

**Brussels, 16.04.1997**
COM(97) 149 final

#### **COMMUNICATION FROM THE COMMISSION**

### **EU RJESEARCH AND TECHNOLOGICAL** **DEVELOPMENT ACTIVITIES:**

###### **COMMISSION'S RESPONSES TO THE RECOMMENDATIONS** **OF THE** **INDEPENDENT EXTERNAL ASSESSMENTS** **IN THE DOMAINS COVERED** **BY** **THE SPECIFIC PROGRAMMES AND THE JRC INSTITUTES** **UNDER THE FOURTH FRAMEWORK PROGRAMME** **AND** **EURATOM FRAMEWORK PROGRAMME**

**COMMISSION OF THE EUROPEAN COMMUNITIES**

Brussels, 16.04.1997
COM(97) 149 final

#### **COMMUNICATION FROM THE COMMISSION**

## **EU RESEARCH AND TECHNOLOGICAL** **DEVELOPMENT ACTIVITIES:**

###### **COMMISSIONS RESPONSES TO THE RECOMMENDATIONS** **OF THE** **INDEPENDENT EXTERNAL ASSESSMENTS** **OF THE LAST FIVE YEARS OF ACTIVITIES** **IN THE DOMAINS COVERED** **BY** **THE SPECIFIC PROGRAMMES AND THE JRC INSTITUTES** **UNDER THE FOURTH FRAMEWORK PROGRAMME** **AND** **EURATOM FRAMEWORK PROGRAMME**

###### **INTRODUCTION**

The Commission welcomes the reports resulting from the 5-year assessment exercises carried out by
high level, independent expert panels. The Commission appreciates the panels' efforts to reach
balanced conclusions and valuable recommendations.

The Council Decision for each Specific Programme [1] requires "an external assessment conducted by
independent experts of the activities carried out within the domains covered by the programme and
their management during the 5 years preceding this assessment". To this end, a new rationalized
evaluation approach has been applied which complies with legislative requirements, takes into
account advice provided by CREST [3], and follows the recommendations of the Commission initiative
(SEM2000) for concrete steps towards best evaluation practices across the Commission's Services [4] .

During the second half of 1996, 18 expert panels have assessed RTD efforts sponsored by the EU
over the previous 5 years under the various Specific Programmes. In addition, 7 independent
visiting groups were established to assess the Joint Research Centre (JRC) activities (i.e. JRC
participation in Specific Programmes and its own research programmes) and another high level
expert to provide an overall evaluation and synthesis of the visiting groups reports. One further
group was charged with providing an assessment covering all Community RTD activities over the
last 5 years (EC and Euratom Framework Programmes). All in all, some 170 senior independent
experts have been involved in the exercise, which subsumes the final evaluations of the programmes
under the Third Framework Programme, producing 18 Specific Programme reports, 7 JRC
Institutes reports and one global JRC report, and one overall Framework Programme report.

This Communication presents the Commission's responses to the major recommendations provided
in the Specific Programme reports. These recommendations are included in the executive summaries
which are reproduced in this document for completeness. The opinions expressed in the executive
summaries are those of the external expert panels and are given under their responsibility. (Each
Specific Programme is presented in a separate section including three parts: I. Executive Summary;
II. Commission's Response; and III. Evaluation Panel. The section on JRC is composed of the
overall evaluation and synthesis report and the list of experts having conducted the assessments of
the seven Institutes.)

The Commission notes in particular the comments made by the expert panels concerning the
management and the procedures of RTD programmes and will, where appropriate, propose

measures.

The full expert reports for the Specific Programmes will be published separately. The results of the
JRC assessments together with the comments of the Board of Governors will be communicated
separately.

The Framework Programmes overall assessment together with the Commission's responses will also
be communicated separately.

1 Council Decisions: 94/801/EC of 23 November 1994; 94/572/EC of 27 July 1994; 94/802/EC of 23 November 1994; 94/571/EC of 27
July 1994; 94/803/EC of 23 November 1994; 94/911/EC of 15 December 1994; 94/804/EC of 23 November 1994; 94/912/EC of 15
December 1994; 94/913/EC of 15 December 1994; 94/805/EC of 23 November 1994; 94/806/EC of 23 November 1994; 94/914/EC of 15
December 1994; 94/915/EC of 15 December 1994; 94/807/EC of 23 November 1994; 94/917/EC of 15 December 1994; 94/916/EC of 15
December 1994; 94/920/Euratom of 15 December 1994; 94/799/Euratom of 8 December 1994; and Council Decisions implemented by the
Joint Research Centre: 94/918/EC of 15 December 1994; 94/919/Euratom of 15 December 1994.

2 COM(96)22() final, 22 May 1996

**3**
CREST advice to Council and the Commission on the monitoring and evaluation procedures for Community research programmes.

**4**
Communication from Commissioners Liikanen and Gradin in agreement with the President, SEC(96)659/5, 29 April 1996.

' For the JRC. the relevant articles of the Decisions do not require a response by the Commission to the experts' reports.

**TABLE OF CONTENTS**

**page**

**1.** **Telematics application** **7**

**2.** **Advanced communication telechnologies and services** **19**

**3.** **Information technologies** **27**

**4.** **Industrial and materials technologies** **43**

**5. Measurements and testing** **55**

**6. Environment and climate** **61**

**7. Marine science and technologies** **73**

**8. Biotechnology** **81**

**9. Biomedicine and health** **91**

**10.** **Agriculture and fisheries, including agro-industry** **103**

**11.** **Non-nuclear energy** **Ill**

**12.** **Transport** **119**

**13.** **Targeted socio-economic research** **125**

**14.** **Cooperation with third countries** **135**

**15.** **Dissemination and optimization of results (Innovation)** **147**

**16.** **Stimulation of training and mobility** **155**

**17.** **Nuclear fission safety** **161**

**18.** **Controlled thermonuclear fusion** **169**

**19.** **Joint Research Center (JRC)** **177**

**Annexes: Terms of reference of the five-year assessment** **191**

**• Annex 1: Specific Programmes** **192**

**• Annex 2: JRC Institutes** **193**

##### **1996 FIVE-YEAR ASSESSMENT** **TELEMATICS APPLICATION**

**I.** **EXECUTIVE SUMMARY**

**Key Conclusions**

**1.** **The** **ICT** **Programme Structure. A single Information and Communication Technologies**
**Programme is recommended, with a major section of it devoted to Applications Systems.** This
would continue on from the Telematics Applications Programme, retaining the successful features of
that programme, notably strong user involvement in real systems development and validation.

**2.** **Vision and Strategic Thinking. The Programme needs to develop stronger vision and strategic**
**thinking to concentrate the limited resources where development work will be most effective.**
**Links with the Policy parts of the Commission need to be strengthened.**

**3.** **The Applications Systems Section Structure.** The worlds of multi-media telematics and networks
have developed considerably since the Telematics Programme was authorised in June 1991, and are
continuing to evolve very fast. The transition from the 4th Framework Programme (FP) to the 5th FP
presents the opportunity to rethink the applications section of the programme in the light of these
technological, market and socio-political developments. **The applications projects should be**
**concentrated on the emerging sectors of new telematics use and on those where there is a**
**particularly strong economic or social case, such as those thrown up by new developments and**
**those related to the information providers part of the "Information Society".**

**4.** **The Application Systems Section Content. In addition to the calls for applications projects**
**organised on a Sector basis, a separate series of calls should aim to create integrated systems with**
**joint usage of two or more different applications, breaking down the Sector boundaries.** In order
to magnify the impact of the programme, **a very important part of the work of all Sectors of the**
**Applications programme should be to build up the physical and intellectual infrastructure,**
**including standards work, and to create platforms that will help to replicate the individual**
**applications projects. Such work should be given priority throughout the Programme.**

**5. Administration, etc.** The "bureaucracy" surrounding this, and other Framework Programmes has
steadily increased to the point where key firms and public user bodies are refusing to participate, and
where it is irresponsible to encourage the participation of the high growth potential, small firms, and
the new applicants we would most wish to see in the programme. **Action must be taken to improve**
**delegation and reduce nugatory paper-work.** Participants in the Programme find their project work
inhibited by the endless demands on their time. For example, measures and projects at both Programme
and Sector level, designed to support the Programme and projects as a whole, seem to the participants
to have risen considerably in the 4th FP, creating an unfortunate impact in the time and effort they
cause to the project teams, which is unlikely to be balanced by the benefits. **These overhead projects**
**should be pruned and normally established only at the request of participants in the applications**
**projects.**

**Introduction**

**1.** **Remit of the Telematics Applications Programme Five Year Assessment Panel**

The Telematics Five-Year Assessment Panel was formed early in 1996 to cover the RTD activities carried
out in the field of "telematics" (that is in the field of applications of computer and/or communication
technologies) during the period from 1991 to early 1996. So the remit of the Panel subsumes the final
review of the Telematics Programme under the 3rd FP (Framework Programme), and the mid-term review
under the 4th FP, into this 5-year assessment. There were 13 members of the Review Panel because of the
complexity and wide coverage of this Telematics Applications Programme. The Panel interviewed the
senior staff and others connected with the running of the programme, and various individual project
participants. Working in sub-groups, Panel members examined the work of the individual sectors.

1.

Questionnaires were sent to all the main participants in the 3rd FP projects (1860) and to the co-ordinators
of such of the 4th FP projects as had been authorised at that time (370).

**2.** **Telematics Applications Programme: History**

The Telematics Applications Programme was preceded by the three exploratory actions, established in
1989 as part of the 2nd FP. The Programme was established as part of the 3rd FP, covering 7 Sectors,
with a budget of 430 Mecu. Four other Sectors were added in the 4th FP, with a total Programme budget
of 912 Mecu. 312 projects were funded under the 3rd FP; so far 422 projects have been funded under the
4th FP.

**3.** **Structure of the Full Report**

The Full Report of the Panel consists of the Key Conclusions, as given above in this shortened version; an
Introduction; an Executive Summary in the form of a Discussion of the Panels Findings; a section
presenting and explaining the Panels' main Conclusions and Recommendations; and a set of the
Conclusions and Recommendations on each of the 12 Sectors of the Programme, with the full Sector
reports given in an Annex. Other Annexes cover the analysis of the questionnaires and finance and project
statistics. This Shortened Report is based on the Discussion section of the full report; the reader is strongly
encouraged to look at the full report to do justice to the conclusions, and to the recommendations
stemming from those conclusions which space prevents setting out in this Shortened Report.

**Discussion of the Key Findings**

**1.** **The Achievements**

The essence of the Telematics Applications Programme lies in Applications; the applications of
information and communications technology to the problems of explicit categories of users, making use of
systems that harness emerging technologies of networking and multimedia to provide solutions. There has
been a significant shift between the 3rd and the 4th FP in the extent and nature of the user involvement in
the programme. True end users can validate the system approach and the human interfaces, and
intermediate users or customer bodies can control or at least influence the subsequent deployment of the
fruits of the R&D work. The Panel found that the majority of the projects in the Programme are firmly
directed to the solution of perceived problems. Especially in the 4th FP, users are strongly involved in
drawing up the workplans, in evaluation of the applications, and in the conduct of the approved projects.
This is much to be welcomed. The so-called "Ten Commandments" of the Programme have proved to be
an excellent way of encouraging highly desirable trends throughout the Programme, and especially in the
encouragement of user involvement. It is too early to have proof that it increases the likelihood of project
success. However, it is a reasonable expectation that true involvement of users in the programme will
improve the deployment of telematics applications.

It has been the great achievement of earlier generations of Commission programmes, like ESPRIT and
RACE, that the academic and industrial research workers of Europe have learnt to co-operate together
through exposure to each other in the work of the programmes. Now the Telematics Applications
Programme is pioneering the coming together of the users of Europe. This may be the great contribution
of the Telematics Applications Programme. Greater coherence in the users, the customers of ICT in
Europe, will make for faster reacting and better informed markets, to the benefit of all concerned.

**2.** **The Changed Environment**

The Telematics Applications Programme has been running for some 5 years. During the last few years the
environment for telematics applications has changed dramatically with the use of the Internet growing
exponentially, and awareness of the potential benefits of telematics becoming widespread. The use of
multimedia has become common-place. Prices of computer hardware and the cost of bandwidth are falling
dramatically, and are predicted to continue to do so. Of course the penetration of multi-media telematics

10

**1.**

has still got a long way to go. The challenge now is both to accelerate the efficient use of such systems,
especially where little penetration has yet been achieved, and to encourage the emergence of integrated
applications.

**3.** **Time for a Rethink**

In the light of the dramatic changes in the market place, the Telematics Applications Programme
sometimes displays a certain tiredness. While not throwing away the distinctive and beneficial features of a
programme that concentrates on real applications with strong user involvement, **the limited resources**
**must be used in a way that maximises positive impact, and concentrates on the areas of most need**
**and where the time-scale of penetration can be best accelerated.** In what is now a global market place,
only quality will lead to success so **projects must involve the European bodies most capable of**
**producing quality work. Some restructuring and re-engineering of the Programme will be required,**
**with the addition to the current main thrust, using applications projects, of new approaches that aim**
**to accelerate the take-up and integration of telematics, and its migration towards multimedia.**

**3.1 The Opportunity for Europe.** The emerging multimedia industries will include some of the largest
business opportunities of the next two decades. Yet, there is no clear picture of how this futuristic industry
will evolve. In the presence of such uncertainty, players who articulate a consistent strategy to exploit the
opportunities and address the risks will shape the evolution of the industry structure. Since Europe
currently lags well behind the market penetration figures for the United States of America, we have an
unique opportunity to develop here a strong, early market for these new applications - upon which world
wide market success can be built. **The Applications Programme must be rethought to take into account**
**this opportunity.**

**3.2 Infrastructure, Standards and Platforms.** It is important to ensure that the necessary infrastructure
is there to enable the current market expansion to continue in an efficient and unfettered manner.
Infrastructure is characterised by the need for standards in the broadest sense; standards of performance or
quality as well as of specification. **The development of open standards must be encouraged to prevent**
**the emergence of de facto standards in the hands of the few. Steps must be taken throughout the**
**programme to emphasise the development of the necessary physical and intellectual infrastructure.**
The presence of many users in the programme creates a unique opportunity to do this effectively if the
projects achieve the right balance between standards, infrastructure, and applications pilots. There is a
marked trend away from bespoke systems to the use of common platform systems that serve a class of
applications. **Ways must be found in the Programme to encourage the development and use of such**
**platforms. The need to encourage work on standards, on infrastructure, and on platforms that help**
**to multiply the impact of the individual projects is so strong that consideration should be given to**
**making it a necessary condition of every project application that it has at least some element of these**
**properties in it.**

**3.3 New Systems for the End-user.** Companies round the world are working on new, simple devices that
are aimed at making it easy to access multimedia applications. With such an "information appliance"
system it will become possible to bring the "unwired masses" into the Information age. New information
appliances will open the way for new services and leisure activities. The successful information appliance
system will come in many flavours, but whatever form it takes, software will be the key component. Ways
**must be found in the new Programme to encourage the development and usage of such appliances.**

**3.4 Focusing and Integration of Applications.** The Programme has shown a welcome flexibility to test
and then implement new Sectors for new constituencies of users. This has led to a growth of the coverage
of the Programme, now with 12 Sectors. The increase in the number of Sectors has brought about a
proliferation of Sector advisory committees, workplan working parties and calls. So, though **it is**
**appropriate to continue to organise much of the applications part of the Programme in Sectors with**
**their own constituencies, the time has come to concentrate the limited resources on: 1) applications in**
**those Sectors where the relative immaturity of current use requires attention; and in those few other**
**Sectors where there is particularly strong economic or social benefit. 2) the integration of**

**11**

**1.**

**applications arising from many constituencies into common systems with many uses and diverse**
**users.** This is likely to require special calls, like the one planned for "Integrated Applications" announced
for December 1996. **And above all: 3) the development of standards, infrastructure, and platforms to**
**encourage accelerated take-up by the emergence of common approaches.**

**3.5 Vision and Strategic Thinking. At all levels of the Programme there is a need for constantly**
**adapting vision and strategic thinking to ensure that the fast changing external scene is properly**
**considered to direct the limited resources to areas and in ways where they can be of most benefit.**
This will require close collaboration with the policy parts of the Commission.

**4.** **The ICT Programme as a Whole**

**4.1 A Single ICT Programme.** For years there has been a degree of friction between the three ICT
programmes. This has not proved to be a major problem, being handled by good co-operation. In
particular the co-operation between Telematics and ESPRIT seems to have improved in recent years, but
the arguments for creating a single ICT programme remain: 1) It is inevitable that there has to be
considerable interaction between the enabling technology and the systems or application work, and gaps
must be avoided.. Some parts of the subject still have much fundamental work to be done on them. There
has to be strong interaction and the boundary will shift all the time. 2) It is undesirable that applicants, and
especially new participants, should be confused about which programme they should be participating in,
and even less desirable that they should play off one part of the staff against others. To provide coherence,
clarity of vision, and drive, a single programme will require a single dynamic, charismatic, and well
informed leader.

**4.2 A Single Management Structure. A single management structure should be created for the 5th**
**FP.** The multiple committees of government officials or nominees do serve a valuable role in keeping the
relevant parts of the national governments well informed. This is particularly important in the area of
applications. By increasing the coherence between the sub-sections, administrative procedures can be
improved. The temptation to impose unnecessarily uniform procedures should be resisted. In determining
the details of the structure one good guide-line is to consider what is the primary constituency that will be
making applications to work on that part of the programme. So **the sub-sections of the Programme**
**might be:**

**1) Long Range Research.** Essentially the old "Basic Research" category but with certain additional fields
stemming from the needs of applications. This would be the constituency of the advanced, basic ICT
research workers in the academic and industrial world. The need for long range, research is as strong as
ever, because of the decline of support in parts of both the public and private sectors in Europe.

**2) Enabling Technology (or Applicable Technology) work.** Essentially the ESPRIT and ACTS
programmes where the constituency is primarily the ICT industries, but without their more user oriented
work where the users come primarily from constituencies other than the ICT industry. This section can be
seen as component or sub-systems work.

**3) Applications Systems Development.** This covers user oriented work for clear constituencies, other
than from the ICT industries; the Telematics stream with user oriented streams from the other
programmes. The work of this sub-section can be seen as system integration and proving work.

**5. Structure of the Applications Systems Section of the Programme**

**5.1 The Need for Sub-Structure. The sub-structure for the Applications Systems section should be**
**based on natural constituencies of users.** Cross sector and integrated applications will need to be
encouraged. It is not desirable to divide public from private applications in conditions, where public
bodies can become private bodies; the difference is not significantly larger than that between certain
constituencies of private sector users.

12

**1.**

**5.2 Reducing the** **Number** **of Sectors.** Too many sectors creates confusion for the inexperienced
applicant. The project that spans the boundaries of Sectors is less likely to find favour with the selection
panels than one that falls squarely in the centre of one Sector's workplan. Yet there are reasons to expect
that a project spanning more than one application field may be particularly deserving of support. So there
is every incentive to reduce the number of Sectors. But, on balance, it is better to retain sectors for users
with a major common interest, than to break down the boundaries and create a continuum with no
distinguishing features. However there should be calls designed to transcend the Sector boundaries. **The**
**Panel recommends that existing Sectors should be merged wherever this makes sense in terms of the**
**convergence of the constituencies, but that this process should not be pushed too far. The primary**
**consideration must lie in the way the constituencies see themselves.** A tightly drawn workplan is the
best way to reduce the over-subscription problem, provided the body that draws up a workplan is well
informed about the priorities for that constituency. This mechanism should also handle the strategic market
survey to detect constituencies that deserve a new part in the programme.

**5.3 Cross Sector Co-operation.** Steps, such as special calls, have to be addressed to encourage crosssector applications and merged or integrated applications. So the Applications sub-programme can be seen
as consisting of a number of Sectors at one time or call, but as a continuum without boundaries at another
time or call. **The majority of the available funds must still be devoted to the normal Sector**
**applications systems projects, but** say **10 - 20%, should be reserved for cross Sector calls.**

**6. The Challenges**

The challenges for the 5th FP Applications Systems Section of the ICT Programme are a mixture of those
that have always challenged the programme managers (such as priority and criteria setting, the evaluation
process in a multi-national environment, project monitoring in a constructive and creative way, and
fostering exploitation while maximising actions like standardisation work); and those new challenges that
now face the Programme manager. Amongst these are:

**6.1 User Involvement.** The involvement of users is a distinctive and successful feature of the Telematics

Applications Programme. However, the involvement of users in projects is not without its problems:

1) Users are frequently inexperienced in participation in R&D projects, unaware of the conventions of the
"Peer Review" process. Some are more used to political methods rather than the scientific approach. This
can lead to problems when political pressure is applied to influence application selection.

2) There is also the danger that the users will be encouraged to become research workers themselves,
which can lead to poor project performance.

3) If the users get too much influence over the workplan and project selection process they can be resistant
to the desirable adoption of innovative new technologies and forward-looking approaches.

4) Most pilot users are large organisations who may wish to tailor the application too tightly to their
specific needs, leading to strong risks of over-specification with resulting limited transferability.

Some ways of minimising these problems are suggested in the main report.

**6.2 Administrative Burden**

**1) In the Process.** It is inevitable that programmes deploying public funds will be hedged about with
regulations designed to safeguard the public purse. But these have steadily grown over recent years to the
point where the programme is being strangled. Many private sector companies are no longer prepared to
participate. The costs of preparing proposals, the delays inherent in project approval and contract
finalisation, combined with the relatively small proportion of applications that can be funded, make it
preferable to forego the advantages in participation. **Delegation must be improved.**

**13**

**1.**

**2) In the Project.** Once a proposal has been approved and a contract signed, enabling the work to
proceed, the research worker finds an ever increasing burden associated with the project reporting and
concertation process, etc. The Commission senior staff claim that some aspects of the burden has been
reduced in the 4th FP, but the workers seem to perceive an overall increase, perhaps due to the spread of
horizontal projects. These distractions represent a severe overhead, involving, as they inevitably do, the
time of senior project staff. **Measures must be taken to reduce such an overhead.**

The small high technology firm or the applicant new to the programme are likely to find the overheads and
delays are unacceptable. Small firms cannot afford to tie up their scarce resources of high quality
manpower in the overhead activities required of a participant in the Framework Programme projects as
they are currently organised. It is irresponsible to encourage such a firm to participate. _Some suggestions_
_for tackling this major challenge of_ _reducing_ _bureaucracy in the Programme are set out in the main report._
It is not, of course, a problem restricted to the Telematics Applications Programme.

**7. Relations with the Policy DGs**

It is highly desirable that the relevant policy DGs should take an active interest in and be closely associated
with the relevant work in an Applications Programme. Special mechanisms been established for certain of
the Sectors. These measures are much to be welcomed, and **such mechanisms should be provided to all**
**Sectors where policy DGs with an interest in the work are to be found.**

**14**

**II.** **COMMISSION'S** **RESPONSE**

The Commission welcomes this independent evaluation of RTD activities on applications of information
and communication technologies in areas of general interest. The recommendations will be taken into
account in orienting the further development of technologies in the Fourth and Fifth Framework
Programmes and in related standardisation and deployment initiatives.

Since telematics RTD activities were started at the European Union level in the Third Framework
Programme, information and communication technologies have developed considerably beyond
recognition (e.g. Internet, CD-ROMs) and their growing convergence implies that developments will
steadily continue. Hence, the transition from the fourth to the Fifth Framework Programme presents the
opportunity to reconsider the Applications component of any future programme concerning information
society related research, in the light of these technological, socio-economic and regulatory developments.
The report of the five-year assessment panel offers very useful paths to explore in order to focus the future
activities on those areas which have the strongest European added value and where there is a genuine
economic and social case.

As the report has clearly noted, during the 3rd Framework Programme the RTD work was devoted
essentially to the development of solutions for the modernisation of services of general interest in Europe.
The 4th Framework Programme has brought further major objectives, notably to prepare the ground for
deployment through public-private and public-public partnerships, and to encourage the opening up of new
markets and the emergence of new companies, in particular innovative SMEs.

The report correctly notes that the transition from the 3rd to the 4th Framework Programme included three
major shifts: from the linear model to the feedback model for research, technological development and
demonstration; from data telematics to multi-media telematics: from technology-push to user-led
orientation. On the latter point, the report stresses the essential contribution made by telematics RTD
activities in developing discreet, topic specific, User Constituencies. Such shifts might have deserved a
stronger emphasis in the report as they resulted in dramatic changes in the work content and in programme
and project management. They should be re-inforced in any subsequent work in the Fifth Framework
Programme, along with other shifts, e.g. increased concentration of RTD effort, stronger contribution to
major EU policy objectives, full implementation of the "contextual research" model (interactive vs.
linear), systematic involvement of users in projects for continuous socio-economic feedback, more
comprehensive coverage of all major techno-economic stages (analysis of user needs, functional
specifications, demonstrator building, laboratory verification, demonstration through field trials,
exploitation plan containing quantifiable economic criteria for future assessment of success).

The Commission welcomes the Panel's view that separate series of calls for proposals should aim at
creating integrated systems with joint usage of two or more different applications, with a view to breaking
down the sector boundaries. Already under the Fourth Framework Programme, the call to be launched in
December 1996 on integrated applications should allow the Programme to draw useful lessons on the
interest, relevance and feasibility of such an approach prior to implementing the next Framework
Programme. These integrated applications are expected to provide user friendly access to various
applications from existing platforms (PC, network computer, Kiosks, personal digital assistant, etc.), to
have an appropriate scale demonstration phase on "digital sites" to allow significant conclusions to be'
drawn in socio-economic terms, and to enable faster move towards the Information Society. The results of
this call and the continuous assessment of project implementation will provide essential information in
support of the preparation of a larger scale operation for integrated applications in the Fifth Framework
Programme.

The Commission takes note of the Panel's recommendation to focus the work throughout the Programme
on building up the physical and intellectual infrastructure, including standardisation work involving
industry, and to create "platform systems" that can be tailored to the individual user requirement whilst
allowing the replication of applications across different sectors. The use of such specific industry platforms
will be encouraged in further RTD activities, drawing usefully on the lessons learned from both the on

15

1.

going sectorial projects and the future integrated applications projects. This approach should certainly
contribute to encouraging cross sector cooperation, making any further programme function as a
continuum of activities without boundaries instead of as a collection of distinct sectors.

The Panel recommendation that preparatory accompanying and support measures as well as "horizontal"
RTD activities within projects like those of the Telematics Engineering sector, deserve a careful study will
be taken up. In the meantime, the Commission does consider these as important actions for supporting the
RTD projects and for fostering the standardisation process and the dissemination and innovation process.
In order to avoid their becoming what the report calls "overheads projects", their optimal weight should of
course be assessed, including through consultation of consortia involved in RTD projects, and their
selection be decided against a clearer strategy and transparent indication of the expected benefits.

The Commission shares the conviction that greater coordination between RTD programmes, in particular
through the concept of research/industry Task Forces, and greater flexibility in programme management
will be beneficial. The "bureaucracy" as it is identified both in the programme management process
(delays inherent in project approval and contract finalisation, large oversubscription, model contract based
on cost statements, etc.) and in the management of projects (reporting, "concertation", etc.), has indeed
been increasingly perceived by participants across the specific programmes as a source of concern and
even sometimes a source of insurmountable difficulties for the management of Consortia. The first
orientations for the Fifth Framework Programme, proposed by the Commission under the title "Inventing
Tomorrow", introduces a simpler structure and more flexible management ideas consistent with the
recommendations of the 5-year assessment.

16

**III.** **EVALUATION PANEL**

**Mr. Knud CHRISTENSEN**

Director, R&D and Production, Scandinavian Mobility International A/S. Vice-Chairman, Centre for
R&D within Rehabilitation (1988-1990). Management consultant with expertise in strategic planning,
productivity and logistic structures (1988-1990).

**Dr.** Max **HERRY**

Owner and Managing Director of Traffic Planning Office Herry, Vienna, and of INVOPLAN, Munich. 20
years experience of national and international transport and telematics projects. Evaluator of 4th FP
Transport programme (DG VII). President of DANUBE (Exchanges between universities and industry in
the EU). Project leader or consortium member of 7 finished EU projects, 4 current EU projects.

**Prof.** **Hannu JAAKKOLA**
Professor for information technology at Tampere University, Finland. Director of research institute for
information technology (1986-1992). Author of some 80 publications on information technology and
technology management.

**Prof.** **Dr. Gunter** **KROËS**

Professor of national economics, Faculty of Urban & Regional Planning, University of Dortmund. Head
of Department for National Economies. Expert on regional development in rural areas. Chairman of the
Telematics Applications programme's external monitoring panel in 1995.

**Dr. Philippe** **LAREDO**
Associate Research Director, Centre de Sociologie de l'Innovation (CSI) at the Ecole des Mines de Paris
(since 1987). Responsible for research in programmes in transport economics, Ministry of Transport,
France (1979-1982). Responsible for national programme on marine technologies and then head of
evaluation unit, Ministry of Research and Technologies (1983-1989). Evaluation of many EU and Eureka
programmes. Expert with numerous papers on R&D programmes evaluation.

**Dr. Louis LEE**

Director of information resource services, University of Westminster (since 1989), with responsibility for
strategic planning, technology and development of academic computing, audio visual resources, libraries
and management information services to the University. Non-executive Director of an SME that supplies
library systems and services. Expert on the use of telematics in the university management environment.

**Dr. Jim MOUNTJOY**
Managing Director, Euristix Ltd. Member of the Irish National Information Society Steering Committee.
Former member ESPRIT Management Committee. Member, Telecommunications Senior Officials
Committe and of European Space Agency.

**Dr. Brian OAKLEY (Co-chairman and secretary of the panel)**
Retired. Chairman of Logica, Cambridge, (1987-1993). Director of the Alvey programme (1983-1987).
Secretary of Science & Engineering Research Council (1979-1983). Member of ESPRIT Management
Committee (1983-1987). Member of ESPRIT Advisory Committee (1987-1992). Chairman of Telematics
programme mid-term review (1993).

**Prof.** **Torleif OLHEDE**

Director of the Department of Health Informatics at Spri, the Swedish Institute for Health Services
Development. Research Associate at the Department of Computer and System Sciences at Stockholm
University and the Royal Institute of Technology. Expert in security informatics. Swedish representative
on Health Care Sector Working Party. Head of group responsible for armed forces strategy for IT,
communications and computer security (1984-1989).

17

**1.**

**Dr.** **Hans-Detlef** **SCHULZ**
Managing Director, Media Transfer GmbH, Darmstadt, Germany. Member of Telematics programme
mid-term review panel (1993). Panel rapporteur, 4th FP call for proposals. Expertise in the field of
telecommunications and telecommunications standards. Rapporteur for multi-media standards in ITU-T,
member of ETSI, ISO, and DA VIC.

**Prof.** **Vincenzo TAGLIASCO**
Full Professor in medical informatics at the University of Genova, Italy. Served as expert for the
Commission on studies of impact of EU R&D programmes, mid-term evaluation of Human Capital and
Mobility programme. Expert and writer on the future socio-economic impact of demographic change and
technology development.

**Dr.** Georges WANET
Member of the Executive Committe of Belgacom since 1992. Head of Corporate Strategy and
Development. Was responsible for strategic planning, data processing, and the FAST project to improve
efficiency of basic telephony services. Also Director of F.N.A., grouping 12 telecommunication operators
to provide managed bandwith services to the Financial Community. Member of Telematics programme
mid-term review panel (1993).

**Mr. Nils WILHJELM (Co-chairman of the Panel)**
Chief Executive of Industriens Realkreditfond (since 1990). Minister for Industry and Trade, Danish
Government (1986-1989). Chairman of Federation of Danish Industries (1983-1986). Deputy Chairman,
Board of Directors of Tele Danmark A/S.

##### **1996 FIVE-YEAR ASSESSMENT** **ADVANCED COMMUNICATION TECHNOLOGIES** **AND SERVICES**

**19**

T. EXECUTIVE SUMMARY

_Europe has excellent_
_first-generation_
_digital_
_communications,_
_and a European_
_information_
_infrastructure is now_
_urgently_ _needed._

_Digital video services_
_should be introduced_

_as quickly as_
_possible.._

_with a coherent plan_
_and timetable for the_
_introduction of a_
_European_
_Information_
_Infrastructure._

_The key objectives of_
_the European R&D_
_have been met, but_
_coherent deployment_
_of multimedia_
_services is slow._

_Further research_

_and technology_
_development is_

_necessary_

With the Euro-ISDN and GSM standards, Europe has an excellent set
of first-generation digital communications infrastructures. In terms of
the technologies used, they are as good as any in the world. However,
it is vital for Europe to continue to develop communication
technologies, standardise for interoperation and introduce a **European**
**Information Infrastructure** combining telecommunications, data
networking and broadcasting capabilities as quickly as possible.

In particular, digital video services should be introduced as quickly as
possible. The technology for digital video and television now exists,
and **a programme to accelerate the penetration of digital** services
**would liberate valuable spectrum for other services.**

Specific features of a European Information Infrastructure need also to
be addressed at European level, taking account of the increasing
importance of the developing Eastern European markets arid the
Mediterranean Countries.

The cost then of providing broadband services to every household in
the EU alone will be enormous. Some progress has been made in the
installation of the core network, but a lot needs still to be done in
extending this to all citizens and small businesses. **Europe badly**
**needs a coherent plan** **and timetable for the introduction of a**
**European Information Infrastructure.**

The key objectives of the European R&D telecommunications
programmes (RACE I / II) have been met. The programme has met its
technological objectives, but the task of technology development is far
from being completed. The ACTS Programme has created an
awareness of the vast potential of digital technology. However, the
deployment of advanced multimedia telecommunication services on the
European market has not reached the level which could have been
expected. The EU RTD has helped interconnectivity and
interoperability, but legislation may be necessary in some cases to
ensure coherent deployment. It is also necessary to improve access to
venture-capital support for innovative high-tech enterprises and start
ups.

Further research and technology development is necessary: It will not
be possible to realise an Information Society in Europe only with the
technologies of today, and technology development must occupy a key
position in future EU RTD. Preference should be given to projects that
lead to improvement of an EII and/or to future concrete commercial
applications, and should mainly include projects that could have major
impact on the Union and its population.

21

_but the focus must_
_now shift from_
_technical standards_

_alone, to trials and_
_volume deployment_

_EU_ _RTD_

_management needs_
_improved_
_contracting.._

_good programme_
_integration..._

The original rationale for European support to RTD in this area still
applies, but the focus must now shift from defining technical standards
alone, to enabling applications and volume deployment on the basis of
unique and widely accepted standards. European standards are a
strategic means to maintain the technological initiative in Europe and to
get a better industrial position in the global marketplace. It is also
essential that the time span between conceptualisation and the
realisation of new developments be shortened. Mechanisms must be
found to achieve real market impact.

In terms of the management of EU RTD, contracting procedures need
to be improved and the benefits of using a results-based contract should
be reviewed, with more emphasis on technical achievement than on
accountancy. Contracting should be faster and more transparent.

Independent advice on programme integration and management should
continue to be sought, and more use should be made of videoconferencing.

_across programmes._ Stronger links should be established with the EU Training and
Mobility, ESPRIT (information technology) and Telematics Application
programmes. There is a need to enhance the transparency and cooperation, including potential joint projects, between these

programmes.

_and with enhanced_

_National initiatives._

_In the 5th F. P.,_

_there should be a_

_focus on the_
_Communication and_

_Information Society,_
_with a coherent_

_strategy and more_
_flexibility._

_Key requirements in_
_communications will_

_be seamless access,_
_ease of access,_
_quality of service,_
_ubiquitous access_
_and service_

_customisation.._

And the concept of National Hosts needs to be re-evaluated to enhance
their use.

Work in the 5th Framework Programme should be around the theme of
RTD for the Communication and Information Society. There must be a
coherent overview of the whole information and communications

technology development area, with a single coherent EU strategy, a
single strategic advisory group and flexibility to move resources within
the whole area as circumstances change. One possibility might be a
Core Technology Programme and Focused Programmes in specific
application areas, as described in Section 6.3.

The main objectives should be to overcome present technological and
infrastructural constraints to the explicit and hidden needs of users and
to stimulate trials to demonstrate "in-the-field" the effectiveness for

users of proposed solutions. It is recommended that use of trial
infrastructures be enhanced in each Member State.

To enable market development, meeting users requirements, the key
requirements are for seamless access; ease of access; quality of
services; ubiquitous access, and service customisation. Particular
efforts must be made to improve access facilities, improve connection
facilities, and optimise the interaction between the "access intelligence"
and the "network intelligence". Standardisation - first for parts and
then the whole - should remain the primary task: ATM systems must
also be completely and finally standardised, and multi-media standards
must be developed jointly by various standardisation groups.

**22**

_**Enabling**_ Other key areas are in optoelectronics/photonics, microwave
_technologies_ _**and**_ propagations, software, interactivity, service engineering, intelligence
_visionary concepts_ in networks, intelligent agents, mobile communications and in
_**must both**_ _**be**_ information security.
_**covered.**_

But visionary concepts must also be addressed, some examples of
which are given in Section 6.5.

**Recommendations**

1. Europe needs a coherent European Information Infrastructure covering all new communication
requirements emerging from telecommunications, data networking and broadcasting; it is the
responsibility of the Commission to take the necessary initiatives.

2. To reach this goal, it is necessary that the process of liberalisation in the telecommunications and
broadcasting infrastructures and services is speeded up.

3. As a counterpart, it is also necessary to strengthen and accelerate standardisation. Only unique
European standards will protect the investments of industry and consumers.

4. Ways must be found to accelerate the volume deployment of digital TV and interactive multi-media
services. In addition, opportunities must be sought for exploitation of global niche markets for highvalue business services.

5. Access to financing for innovation must be improved. The Commission should take initiatives to
create a better venture-capital financing environment; some options are indicated in Section 1 of the
Report.

6. A flexible and functionally effective approach to EU RTD is needed from all European Institutions.
The Commission must ensure co-ordination, co-operation and transparency in their management of
programmes. In advanced communications, good programme integration should be continued, and
with more co-operation with other programmes.

7. We must prepare now for the 5th Framework Programme, with appropriate and flexible provisions
for Information and Communications Technology RTD; some of the key issues are highlighted in
this Report and exploratory work in these areas should already be started.

8. It is vital to include visionary concepts, as a means of enhancing value chains, and as the seeds of
future European competitiveness.

**23**

II, COMMISSION'S RESPONSE

The Commission welcomes this independent evaluation of EU RTD in the broader perspective of advanced
communications developments in Europe. The recommendations will be taken into account in orienting the
further development of technologies in the 4th and 5th Framework Programmes and in related
standardisation and investment stimulation initiatives.

The Commission accepts its responsibilities for catalysing the development of a coherent European
Information Infrastructure covering all new communications requirements and addressing the specific
features of European regions, Eastern European and Mediterranean countries markets. The technology
developments in the RACE (1988-1994) and ACTS (1994-1998) programmes are directed to support interoperability in the convergence of telecommunications, data networking and broadcasting areas. In
addition, coherence and interoperability are key features in the new regulatory framework for
communications in the Information Society.

Liberalisation and competition in provision of infrastructure and services has been a key objective of EU
telecommunications policy in recent years. The Commission will seek to ensure that the process of
liberalisation in telecommunications is followed through by Member States on the agreed timetable.

The multiplicity of infrastructure and service providers in a competitive market, and the new diversity of
technologies for service provision, will increase the need for effective standardisation and consensus
development. The EU RTD programmes, both RACE and ACTS, have made major contributions to
technical standardisation - as noted in the 5-year assessment. Further support will be provided from the
remainder of the ACTS programme in the form of guidelines and contributions to industry-led
standardisation. The need for substantial EU support to standardisation will be considered in preparing the
Commission's proposal for Information Society RTD in the 5th Framework Programme.

The volume deployment of digital TV and interactive multi-media services is primarily a responsibility of
the private sector. Nevertheless, the new framework for EU investment stimulation in Transeuropean
Networks will be used by the Commission to catalyse the coherent and simultaneous roll-out of new
services within the EU. Exploitation of global niche markets for high-value business services will be
facilitated by emerging "electronic commerce" systems. The Commission will implement a new Coordinated Electronic Commerce Initiative in 1997, building on the RTD in the ACTS programme and the
Memorandum of Understanding recently signed by leading industrial interests and SME associations.

In 1996, the Commission published a Green Paper on Innovation, and has adopted an Action Plan on the
basis of the reactions from Industry and Member States. The Green Paper covered the improvement of
innovation financing, particularly for new technology-based firms. The options in the Strategic Audit
Report will be given careful consideration.

The Commission shares the conviction that greater co-ordination between RTD programmes, and greater
flexibility in programme management will be beneficial. The first orientations for the 5th Framework
Programme, proposed by the Commission as "Inventing tomorrow", introduce a simpler structure and
more flexible management ideas consistent with the recommendations in this "Strategic Audit". A reinforced provision for visionary research at European level will allow many of the new ideas in this report
to be taken forward.

COM(96)332 Final of 10.07.96.

24

**Ill,** **EVALUATION PANEL**

**Mr. Stelios Argyros MEP**
As a Member of the European Parliament, Stelios Argyros sits on the Committee on Research,
Technological Development and Energy and the Committee on Economic and Monetary Affairs and
Industrial Policy. He has served on the Board of numerous companies in Greece, has been Chairman of
the Board and President of the Federation of Greek Industries and is currently a Vice President of UNICE.

**Mr. Max Artigalas**
General Manager Recording Programme Administration of THOMSON multimedia. After being technical
manager of Thomson Broadcast, he joined THOMSON multimedia where he is also recording domain
manager for Corporate Research and Advanced Development.

**Dr. Hans Baur**

Former Executive Vice President of Siemens AG, and now acting as an advisor to several national and
international bodies and to Mr. Bangemann on the Information Society. During his career with Siemens
AG, Dr. Baur was a member of the Corporate Executive Office and was responsible for the business
groups dealing with telecommunications and electronic components. He also served as President of the
Supervisory Board of Siemens Nixdorf Informationssystems AG.

**Mr. Umberto de Julio**
Co-General Manager of STET, with responsibility for strategic planning, technology and development of
new service, prior to joining STET in 1995, Mr. de Julio held the position of Director of the Network of
Telecom Italia in 1994. His earlier career with SIP began in 1972, where, in 1986, he was appointed
Director of the Network. He is a Member of the Board of Telecom Italia, Telecom Italia Mobile, CSELT,
STREAM, STET International and is also President of the General Assembly of EURESCOM and ViceGeneral President of AEI. Mr. de Julio was assisted by Mr. Giovanni Perucca, Director of R&D of
STET.

**Mr. Aimo Eloholma**

Vice President of Telecom Finland Ltd., Sales and Marketing, since 1995, Mr. Eloholma was formerly
Vice President of Telecom Finland, Business Services, and has held a variety of positions within the PTT
in Finland. He is Chairman of the Board of Tele Yritysviestintà Oy (provider of PBX's) and Yritysverkot
Oy (Corporate Networks Ltd.) and serves on a number of Committees in Finland. He is also a member of
Information Society Forum (European Commission). Mr. Eloholma was assisted by Mr. Timo Rajamaki,
Director of the Research Centre of Telecom Finland.

**Dr. Francisco Pinto** **Balsemâo**

Chairman and majority share holder of IMPRESA, S.G.P.S., a holding company with major positions in
newspapers, printing, advertising and a Portuguese private television channel. He is Professor of
Communication Science at the New University of Lisbon, Chairman of the European Institute for the
Media, Vice-President of "Fondation Journalistes en Europe" and Member of the Board of the Media
Business School. He was one of the founders (1974) of the Popular Democratic Party (PPD), now the
Social Democratic Party (PSD) and from 1981 to 1983 he was Prime Minister of Portugal. Dr. Pinto
Balsemâo was assisted by Mr. Antonio Trigo de Sousa, Technical Director in SIC.

Dr. **George** T. Waters

Director of the Technical Department of the European Broadcasting Union since 1986, Dr. Waters has
been involved in broadcasting for more than 35 years. He completed a seven-year term as Director
General of RTE, Ireland's national broadcasting organisation, and was a Director of the International
Council of the Academy of Television Arts and Sciences (USA) and Vice President of the European
Broadcasting Union. In 1993 he was appointed President of the International Academy of Broadcasting
(IAB).

**25**

##### **1996 FIVE-YEAR ASSESSMENT** **INFORMATION TECHNOLOGIES**

**27**

**I.** **EXECUTIVE SUMMARY**

**1.0** **Scope of the Assessment: Lessons for the Fifth Framework Programme**

The Panel's five-year assessment of the European Strategic Programme for Research in Information
Technologies (ESPRIT) is based on an examination of the many documentary materials supplied by the
Commission, including the 1996 report of the ESPRIT Review Board produced under the chairmanship of
Professor Umberto Colombo. It has also interviewed senior Commission staff concerned with Information

and Communications Technology (ICT) policies. In addition, the Panel sought the opinions of national
delegates to ESPRIT.

The review has focused on activities carried out under the European Community's Third Framework
Programme, which ran from 1991 to 1994, and during the initial period of the Fourth Framework
Programme, up to early 1996. This assessment incorporates the final evaluation under the Third
Framework Programme.

The Panel has especially focused on some specific features of the present ESPRIT programme and, in
agreement with the Commission, on strategic issues affecting ICT research and technology development in
the Fifth Framework Programme. It has arrived at a significant number of constructive recommendations
that reflect the Panel's view of the crucial role of ICT and of the importance of the Information Society. It
hopes that the Commission will endorse this judgement and assign a budget to the future ICT programme
in the Fifth Framework Programme which is at least equivalent to the present one; that is, about one third
of the overall Framework Programme Budget.

**2.0** **A Brief History of ESPRIT: Steadily Widening Objectives**

Community funding of IT R&D begun in 1984 . During the first decade, the programme has been
essentially aimed at closing the gap between Europe's information technologies industries and those of the
US and Japan.

The strategy changed radically with the launching of the Fourth Framework Programme in 1994 which
recognised the increasingly critical role played by IT in the competitiveness of all industry. The new IT
Programme:

promotes vertical collaboration between ICT users and suppliers;

gives much closer attention to focusing results on market needs;

facilitates access to new technologies and their take-up;

encourages an awareness of emerging R&D results;

employs a number of new tools to implement the programme such as networks of excellence and
focused clusters (activities covering different technological areas with a single well-defined goal).

Council decision 84/30 (28.02.84) established ESPRIT as part of the First Framework Programme (1984 - 1988)
with a budget of ECU 1,500 million. Council decision 88/279 (11.04.88) continued ESPRIT into the Second
Framework Programme with a budget of ECU 3,200 million and Council decision 91/2328 (15.07.91) renewed
ESPRIT as part of the Third Framework Programme with a budget of ECU 2.035 million. Council decision
94/802 (23/11/94) established a new programme, called Information Technology Programme, as part of the Fourth
Framework Programme with a budget of ECU 1,911 million.

29

**3.**

**3.0** **The Context: Europe's Transition towards the Information Society**

The Panel's report is written against the background of rapid change in the importance, development and
use of ICT. The moment is opportune to present some new ideas enabling Europe to adapt to at least four
key developments:

 - Europe's transition to the Information Society has begun and is a crucial challenge for Europe's future
economy and society;

 - deregulation of European telecommunications, which should be a major factor aiding this transition,
is now little more than a year away;

 - rapid growth in use of the Internet is bringing new opportunities to businesses and citizens;

 - multimedia technologies are penetrating the worlds of business, the media, public services and
education.

This report urges radical changes in the organisation and design of ICT programmes to enable the EU to
respond better to these developments.

**4.0** **Priority Issues: A Single ICT Programme Which Brings More Products to Market**

Two priorities have been identified and addressed in the Panel's recommendations:

**4.1** **The need to merge the ACTS, Telematics and ESPRIT programmes**

The Panel has come to the conclusion that the objectives of these separate programmes would be better
achieved if they were brought together in a single programme that covers all aspects of ICT required by
the Information Society.
Given the fact that ESPRIT operates on the basis of annual work programmes, the Panel believes that
many of the changes in this direction can be implemented on a step by step basis during the remainder of
the Fourth Framework Programme. This will help to smooth the transition to the Fifth which begins in
1999.
An attempt to define a timetable for the programmes' merger would have been outside our terms of
reference. But we believe it would be helpful to establish a joint coordination committee between the three
programmes to prepare for the event. We are aware that we are advocating a profound change which
needs careful planning and, above all, avoidance of administrative obstacles to what would be a significant
improvement in Europe's programmes for ICT.

**4.2** **The need to achieve much swifter commercial applications of research results**

The panel has concluded that ESPRIT's programme objectives have been satisfactorily accomplished.
Among other things, it has made a vital contribution to promoting cross-border collaboration between
small and large industries, universities and research institutes.

However, the programme could be even more effective if - in addition to activities such as best practice,
first-user actions and assessments - better mechanisms were established for ensuring the commercial
exploitation of results. A number of actions could be envisaged, including:

- focusing results much more explicitly on the tasks needed to construct Europe's Information Society;

- initiatives to promote the commercial exploitation of research results;

**30**

3.

- developing the tools to measure and monitor European ICT development, take-up and market shares;

- encouraging SMEs with innovative capacities to strengthen and speed up their marketing of new
value-added products;

- improving the efficiency and accountability of the programme's administration, partly by making it
more results-oriented;

- define a long-term strategy for European research and technological development in ICT.

**5.0** **The Panel's Main Findings**

The Panel's findings look back to the five years of the assessment period and forward to the needs of the
future programme.

**5.1** **Assessment Period 1991-1996**

**5.1.1 Programme Objectives: Time to Add The Needs of Society At Large**

The Panel agreed that ESPRIT's objectives have been relevant and fully endorsed its new focus on
responding to the needs of business and industry as a means of boosting competitiveness and employment.
However, there is now an additional need to broaden the objectives to include the needs of society at large
which must find their response in the evolving infrastructures, services and applications of the
Information Society.

Having assessed the programme's performance against its objectives, we would draw particular attention

to:

- the positive change in the dynamics and perspective of ESPRIT from the Third to the Fourth
Framework Programmes;

- the focus on relevant topics has been made possible by a broad description of research tasks and a
continuously updated work programme;

- an imbalance between the user-need, market-driven priorities and the requirement for long-term
research. The new focus is welcome but we are concerned that the necessary balance with long termresearch is being jeopardised.

**5.1.2 Programme Efficiency: Relevant Effort, Still Some Shortcomings**

The Panel considers that the management as a whole has made great efforts to raise the programme's
performance and efficiency. In addition, worthwhile changes have been introduced to make the
Programme more flexible - a feature which must be characteristic of the Fifth Framework Programme.

However, the search for flexibility has turned on attempts to satisfy the specific needs of proposers and
this, paradoxically, has made certain aspects of the programme very complex (e.g. variety of actions and
information packages, different types of proposal submissions).

The Panel also wishes to draw attention to the more general problem of administrative delays which
characterise the Framework Programme as a whole, particularly in concluding contract negotiations and
making payments.

**5.2** **The Future of ESPRIT: Five Challenges for the Fifth Framework Programme**

The Panel has identified five areas that must be addressed in the Fifth Framework Programme:

31

3.

(i) Programme Strategy and Design

(ii) Programme management and evaluation

(iii) The Information Society

(iv) Competitiveness and exploitation of research

(v) SMEs and start-up companies

**5.2.1 Programme Strategy and Design: Three Into One** **Will** **Go**

Currently, there are many overlaps between ESPRIT, ACTS and Telematics which need to be turned into
synergies. This could best be achieved by merging these three ICT programmes into one. Within this
single programme a proper balance must be struck between development of close-to-market innovative
applications and long-term research.

The Panel's detailed recommendations envisage groupings of applications and technologies within one ICT
programme. This would have two clear purposes: to stimulate innovative applications of emerging
technologies and to satisfy the requirement for strategic and long-term research.

**5.2.2 Programme Management and Evaluation: In Need of Overhaul**

Every assessment of the Framework Programme and of its constituent elements as well as a survey of
national opinion undertaken by the Panel confirms that the programme management has become more
efficient. Nevertheless, the Panel supports the widespread view that there is still considerable room for
improvement.

Oversubscription is a familiar problem with Community-supported initiatives and it imposes heavy costs
and burdens on the proposers. It is important to resolve the difficulty because it causes frustration and
dissatisfaction and could prompt dynamic companies to abandon their interest. The limited introduction in
ESPRIT of the two-step procedure has proved an extremely useful attempt to deal with the situation . The
Panel recommends extending it to the whole programme.

The Panel's detailed recommendations seek to make the future ICT programme an efficient instrument for
Europe's ICT community in which delays would be cut by a half, and project performances much more
strictly evaluated.

**5.2.3 The Information Society: ICT as a Tool for Economic Growth, Social Cohesion and**
**Personal Development**

As the Information Society evolves, key aspects of political, economic and social life will be increasingly
permeated by the use of ICT and its applications. If Europe handles the transition in a timely fashion, it
will significantly strengthen its competitiveness in global markets and cash in on faster economic growth,
more employment and a better quality of life.

By contrast, if it continues to move more slowly than some of its competitors, then our countries will find
it very difficult to maintain, let alone increase competitiveness across the broad range of their economies.

A very serious obstacle to the realisation of the Information Society is that European businesses and
individual citizens are lagging far behind the US and some other countries in embracing ICT and in
exploiting the opportunities it offers for both corporate and personal growth. The need to integrate the
use of ICT into education and training is paramount, both to establish computer literacy and as a tool for
learning.

**32**

**3.**

The Panel's detailed recommendations seek to ensure that the future ICT programme is a key pillar in the
construction of the Information Society. It should support projects which help to deliver public goods
serving citizens' needs, which act as a catalyst for user-friendly applications and which foster a growing
market for the new technologies.

**5.2.4 Competitiveness and Exploitation of Research: The Former Grows if the Latter**
**Succeeds**

Innovative development and innovative use of ICT has to be both visionary and focused if Europe is to
maintain and enhance its competitiveness in the global market place. Competitiveness creates employment
and both can be secured if Europe nurtures the development and take-up of a world-class ICT that both
produces and uses ICT to offer constantly improved applications and services.

Constant improvement would result from a better commercial exploitation of the results of ESPRIT
projects. In cases where an attempt is made, windows of opportunity are frequently missed because the
process has been far too slow to adjust to rapid changes in the market place. The Panel believes that the
existing eligibility criteria, combined with various practices, has sometimes led to the creation of
somewhat artificial teams lacking clear commercial objectives. Relaxing some of the criteria and insisting
on common commercial objectives would certainly improve the situation.

The Panel's detailed recommendations aim at more benchmarking of ICT-based competitiveness in the
future programme, measures to ensure a shorter time-to-market, encouragement of projects which have
an entrepreneurial final objective and supporting measures to help commercial exploitation of results.

**5.2.5 SMEs and Start-up Companies: Finding the Right Responses to Particular Needs.**

Small and medium-sized enterprises account for around 80% of Europe's employment. ESPRIT is well
able to provide specific support to SMEs and start-up companies, but its full potential is not being
exploited.

Even though the SMEs represent 30% of the participants, the programme would benefit from more
simplicity to encourage SMEs that have made serious efforts to develop commercial ICT applications. Too
few awards are made to SMEs to enable the sector to achieve the commercial opportunities open to it.

The Panel's detailed recommendations seek to endow the future ICT programme with procedures which
are more user-friendly for SMEs, structures which are more supportive, funding rules which are more
flexible and special measures for encouraging risk taking and for helping start-up companies.

**Recommendations**

All of the following recommendations have been designed to apply to the future single ICT programme,
except for group A which concern the Framework Programme as a whole.

**A.** **Improving the Framework**

**A.l** **Cutting Delays**

a) The Commission should overhaul its current administrative functions and re-engineer key
procedures in order to cut delays by half. Payment procedures in particular must be
streamlined to reduce the time from submission of an invoice to actual payment from the
current 90 days to 30 days.

**33**

**3.**

b) The key administrative procedures should be reviewed to speed up decision making by
delegation of authority to appropriate levels and by reduction in the layers of decisions.

c) The project partners should be allowed to charge for expenditures from the day of the
project's approval so as to cover their costs during the contract negotiation phase. A down
payment should be made immediately upon contract signature.

d) The current payment scheme based on statement of costs should be abandoned in favour of a
deliverables-based payment scheme. This will not only simplify procedures but also
encourage more efficient and productive projects.

e) In order to raise efficiency, timeliness and effectiveness, some of the non-strategic
management tasks undertaken by Commission staff should be wholly or partially contracted
out to competent external organisations.

**A.2** **Effective post-project evaluation.**

A simple method must be developed to evaluate projects after their completion, and a user's guide must
describe how and when to apply this method to different types of projects. Contractors must be obliged to
provide the Commission with the necessary information to permit effective post-project evaluation.

**B.** **Programme Strategy and Design**

**B.l** **Uniting Strengths.**

a) Separate programmes for Telematics, ACTS and ESPRIT should be abandoned and be
replaced with groupings of applications and technologies under one integrated ICT
programme. This is what is necessary to bring together all the technologies required to build
the Information Society.

b) In allocating funds in the Fifth Framework Programme, the Panel strongly recommends to
assign a budget in line with the total ICT programmes in FPIV and not below one third of the
overall FPV budget.

**B.2** **Facilitating** New **Ideas.**

a) The new ICT programme must emphasise the need for Europe to become more innovative by
stimulating the development of innovative applications using emerging and enabling
technologies.

b) Exchanges of researchers, corporate engineers and professionals from hospitals, schools,
public services, etc., should be encouraged to create a better understanding between
technology inventors, developers, and users. This would improve the rapid application of new
ICT developments.

**B.3** **Incorporating Vision.**

a) A strategic vision of the future organisation and functioning of Europe's Information Society
must be constantly developed and reviewed.

b) An "ICT Foresight Board" should be set up to help develop a common European vision on
long-term research and market trends.

**34**

**3.**

c) Long-term research plays an important part in the shaping of Europe's future and must be
given a secured place in European research and technology development in ICT. Around 15
% of the programme budget should be reserved for basic long-term research, strategic
research, and radical innovation.

d) As the scope of the future Framework Programme broadens to address the needs of society at
large, the European Parliament and the Council of Ministers should play a more active role
in defining strategic goals, evaluation criteria and methods, assessing the results of
programme evaluations, and steering the course of the programme.

**C.** **Programme Management and Evaluation**

**C.l** **Serving the Client.**

a) The future ICT programme must be a more efficient instrument to the European community
of ICT users and suppliers.

b) To reduce the overwhelming volume of applications, the two-step proposal, introduced in a
limited number of areas of the programme, should be the norm for all areas. This should be
based on a short 2 -4 pages pre-proposal which would be followed by a full proposal only if
the pre-proposal was accepted.

c) The Focused Clusters, introduced for the first time in ESPRIT under FPIV, should be more
widely used in the next ICT programme.

**C.2** **Increasing Accountability.**

a) The future Framework Programme should set specific goals for European ICT development,
take-up and market shares, and for a target growth in employment.

b) Indicative measures of how the programme is succeeding in building the Information Society
should be established to demonstrate the effectiveness of the programme to its stake-holders:
the Member States, the regions and their citizens.

**D.** **The Information Society**

**D.l.** **Enabling the Information Society.**

a) An Information Society Cluster should be organised within the future ICT programme for
promoting the development and take-up of IS technology in such areas as content creation,
information management, applications for citizens, communities and companies. The Cluster
should be allowed to receive ideas and project proposals on a continuous basis.

b) Proposals should be given a priority that bring together ICT suppliers and users in joint
efforts to define, develop, deliver and take up IS technology and services.

c) A single point of contact for Information Society projects should be established at the
Commission with particular responsibility for coordinating the allocation of ICT programme
finance with the structural funds.

**35**

**3.**

**D.2.** **Networking Citizens and Communities.**

a) Pilot projects should be identified to foster understanding and acceptance of the Information
Society and to encourage the use of ICT in every day life. Special attention should be given to
those who may have particular difficulties of access to technology or in using it, such as the
elderly, the unemployed and the handicapped.

b) Closer links aimed at promoting the use and understanding of ICT must be established
between the programme and authorities responsible for education and training in Europe,
typically at the regional level.

c) National and regional authorities together with non-governmental and professional
organisations (e.g. teachers and doctors) should be encouraged to submit project proposals,
preferably in vertically integrated teams involving all players in the ICT value chain, from
suppliers to users.

**D.** **3 Preparing the Citizen for the Information Society**

a) Strong encouragement should be given to specific projects targeting employment issues, such
as training, retraining and life-long learning technologies or applications. In addition, the
likely impact of a proposed project on employment should be included in the evaluation
criteria.

b) The future ICT programme should support the establishment of appropriate technical
standards not only to encourage business and market development but also to protect citizens'
rights, particularly in the fields of the multimedia content industry and electronic commerce.

c) The programme shall encourage and support projects that specifically address the issues of
low-cost access to public information.

d) The future ICT programme must encourage and support projects aimed at improving the user

interfaces of ICT applications in general, and of training systems in particular.

**E.** **Competitiveness and Exploitation of ICT**

**E.l.** **Establishing Milestones**

a) A long-term strategy for achieving and sustaining European competitiveness through the
development and take-up of ICT must be developed by the Commission in collaboration with
industry. The strategy must include clear objectives, market-share targets, and the expected
contribution from RTD. A unit should be set up to monitor how Europe is progressing
towards its goals.

**E.2** **Shortening Time to Global Markets.**

a) The programme must encourage industry to develop and bring ICT applications to the market
in shorter time.

b) It must emphasise collaborative efforts in commercial exploitation and facilitate a marketdriven approach by shortening the life of projects so that time-to-market periods are between
1-2 years.

36

**3.**

**E.3** **Selecting the Fittest.**

a) The programme must support those who can and will exploit the results of European funded
R&D, and discourage those partners who have no real interest or capacity for exploitation.

b) More efficient models of cooperation should be encouraged, with smaller partnerships built
around real interests. Every partner in a team must have a distinctive and necessary role. In
evaluating proposals, the team efficiency and the potential for commercial exploitation of the
project results must be given higher weighting.

**E.4** **Moving Exploitation to the Top of the Agenda.**

a) The Commission should set up a Help Desk including a network of experts in commercial
applications, marketing, industrial and trade law, banking and venture capital to provide
coordinated support for commercial exploitation and to encourage benchmarking.

b) An Innovation Financing Help Desk should be set up and linked to the Framework
Programme as proposed in the Commission's Action Plan for Innovation.

c) The Commission should foster the creation of an advisory panel involving major players with
a decisive influence on the development of an effective single European ICT market (utilities,
PTTs, regulators etc.).

d) The Commission should establish and fund Reference Sites where European applications can
be demonstrated. Such demonstrations must be well publicised and accessible via the WWW.
Sites to facilitate local access should be established in large potential markets in the US,
Japan and the Far East.

e) Priority must be given to achieving a more rapid penetration of world-wide markets by
helping companies to set up subsidiaries and joint ventures in developed markets. These
should be treated as start-up companies for the purpose of RTD in the new Framework
Programme, provided they bring employment to Europe.

**F.** **SMEs and Start-up Companies**

**F.l** **Making Life Easier.**

a) The Commission should establish an SME Desk as a single point of contact between SMEs
and the programme. The purpose of the Desk should be to provide full support and to
eliminate the present difficulties companies have in navigating through the programmes. The
Desk would evaluate ideas for proposals and provide all of the necessary information on what
is required of a proposer and when.

b) Create a new category of Small and Medium-sized Project (SMP) aimed mainly, but not
exclusively, at SMEs which runs for less than one year and with Commission funding not
exceeding ECU 500,000 per project.

c) Fast-track support should be established for Small and Medium-sized Projects, allowing
adoption of unsolicited proposals not linked to a particular call. The services of the SME
Desk would be at the disposal of SMEs wishing to pursue a small project.

d) Some future ICT programme funding - around 5% - should be earmarked for special calls
aimed at stimulating partnerships in which SMEs take the leading role in the development and

**37**

**3.**

exploitation of results. The adequacy of the funding should be judged in the light of the
scheme's success in creating these highly desirable partnerships.

e) Efforts should be made to thoroughly familiarise SMEs with the future ICT programme and
its objectives by involving them more closely in the consultation process and by extensive use
of external evaluators and project reviewers drawn from SMEs.

**F.2** **Matching Market Endorsement.**

a) Programme funding must be made more flexible in order to provide specific incentives to
SMEs with exploitable ideas.

b) Funding should be increased to 60% when the potential commercial validity of an exploitable
project has been endorsed by venture capital backing, or by a large company covering at least
10% of a project's costs without itself participating.

**F.3** **Stimulating Start-Ups.**

a) Maximum funding of 75 % should be available for projects with good exploitation prospects
involving start-up companies which are not subsidiaries of other companies.

b) Grants should be available to start-up companies which are partners in successful projects to
cover the costs of finding and securing investment capital for commercial exploitation. These
could be 10 - 15 % of the amount that the EU contributed to the start-up's part of the project.

38

IL COMMISSION'S RESPONSE

The independent evaluation of the programme over the last five years was prepared by the high level
group of experts chaired by Professor Roberto Carneiro. The evaluation was based on the examination of
documentary material provided by the Commission including the final report produced under the
chairmanship of Professor Umberto Colombo which evaluated the activities supported by the programme
under the previous Framework. The Commission welcomes the recommendations of the panel which
essentially address the future orientations of the programme.

The Commission views the main findings and recommendations as generally useful for the remaining
period of the current Framework, and for acting as guidelines for the current discussions associated with
the preparation of the 5th Framework Programme. The Commission welcomes the recommendation that a
number of new approaches introduced in the current programme (e.g. rolling workplan, regular calls,
two-step evaluation) be generalized as appropriate in the Fifth Framework Programme.

The need for combining various technologies and actions towards the establishment of an "user-friendly
Information Society" has been stressed in the Communication "Inventing Tomorrow". The Commission
welcomes the panel's views on this topic and shares its conviction that greater convergence of efforts is
necessary and that a single and integrated programme addressing Information and Communication
Technologies is required. As suggested in the executive summary, the programme will take advantage of
its rolling workprogramme to start implementing changes during the Fourth Framework Programme to
smooth the transition into the fifth.

The Commission supports in particular the view that technology take-up should be further emphasised in
the forthcoming programme to imrove competitiveness and create employment through a more widespread exploitation of ICT. The Commission welcomes the panel's emphasis on the role of citizens and
communities. Technological progress in the ICT area increasingly depends on dynamic markets for new
products and services and hence on the ability and willingness of citizens and workers to use new ICT.
The balance between long-term research and take-up activities needs to be defined according to the
industrial and socio-economic objectives of the forthcoming programme. A strategic and forward-looking
vision of research and social developments is certainly required, as mentioned in the "action plan for
innovation" to identify promising technologies and markets.

The Commission welcomes the recommendations aiming at improving SMEs' access to the programme.
The suggestions made by the panel support the "Inventing Tomorrow" proposal to help SMEs access the
programme by developing technology transfer mechanisms and by establishing links with venture capital
and financial engineering tools.

Finally, the Commission supports the view that, at the Framework level, additional efforts need to be
made to improve the flexibility associated with, and the efficiency of, administrative procedures, to
shorten the deadlines and administrative costs so that dynamic companies, in particular SMEs, are further
encouraged to join the future programme.

39

**TÏT.** **EVALUATION PANEL**

**Prof.** **Roberto Carneiro (Chairman),** _President of TVI, Televisào Independent S.A. (Board of_
_Shareholders) Group Forum, SGPS._
Former Minister of Education and Sports in Portugal (1987-91), senior World Bank (IBRD and IDA)
consultant for projects in Latin America and Africa (1975-87); and since 1978 OECD senior consultant on
education policy, manpower planning and public management; and UNESC senior consultant and expert
on education and training policy. Recent international activities include consultant coordinator to SIGMA,
EEC/OECD joint venture designed to implement State reform in Central and Eastern European Countries
(1992-93); OECD Examiner of the French Educational Policy (1993); Member of the International
Commission for Education in the 21st Century (UNESCO, 93/95); vice-president of the Information
Society Forum and Member of the Bureau which coordinates the European Commission Study Group on
Education and Training under the auspices of Commissioner E. Cresson.

**Andrew** **Bos** **well,** _Managing Director, Enterprise Technology and_ _ICL_ _Technical_ _Director._
Chairman of the UK National Information Infrastructure Task Force, which works with the Department of
Trade and Industry on ICT related issues. Appointed to the G7 Committee responsible for accelerating
standardisation in ICT, an initiative that stemmed from the G7 summit of 1995.

**Jean-Michel** **Chassériaux,** _Managing Director_ _ofERCIM_ _and Director of International_ _Relations_

_atlNRIA._

Scientific attaché to the French Embassy in London (1973-77), senior consultant for the Ministry of
Foreign affairs and the Ministry of Industry (1977-84). Deputy-director for training, research and regional
affairs at the ICT industries' Unit of the Ministry for Industry (1984-87). Director for international affairs
at the Ministry of Research and Space (1988-93). Professor at the Paris 7 University since 1983.

**Bernt Ericson,** _Vice-President, Research and Technology, Telefonaktiebolaget LM Ericsson._
Joined Ericsson in 1969 as Chief Engineer and member of the Business Strategy Group; Department
Manager for special projects within the Strategic Product Planning division; Director of applied Research
and finally, vice-president. Elected as member of the Royal Swedish Academy of Engineering Sciences
and of the European Science and Technology Association(ESTA) of the European Commission.

**Roberto Galimberti,** _Chairman and CEO of Etnoteam._
From 1968 to 1973 head of Digital Computing Division of Laben. With Italtel as Research and
Development Director and later as head of the Toll Switching Division. Since 1982 Chairman and CEO of
Etnoteam. Since 1994 Chairman and CEO ofl.NET, a leader in Internet business connectivity. Also vicepresident of CED Borsa (Italian Stock Exchange Information System Company), and of Intercai Etnoteam.
Member of the board of Anasin (Italian Computer Services Association).

**Hartmut** **Raffler,** _Senior Director, Siemens Corporate Technology_ _Division._
Joined Siemens AG in 1979. Responsible for the Innovation Project Communications. From 1993 to 1996
Head of the Software and Engineering Department. Since April 1996 Head of the Information and
Communications Department. Main topics: Information and Media Technologies, Networks and Video
Communications, Security in Systems, Knowledge Processing and Man-Machine Cooperation.

**Josep Maria Vila Solanes,** _General Manager of INDRA Group, President of INDRA SSI and_

_President of INDRA SCA._
Chief Executive Officer of ERITEL, General Manager of Olympic Games Organising Committee at
Barcelona '92. Deputy Manager of ENHER and Branch Manager of UNISYS. Director of the
DATAMATION magazine (Spanish edition). Professor in the Nuclear Technology Department,
Universidad Politécnica de Barcelona, and formerly Professor in other universities and research centres in

Barcelona.

**40**

**3.**

**Herbert Wotke,** _Managing Director of the Austrian Industrial Research Promotion Fund (FFF)._
Served for more than 2 decades in the FFF (Forschungs Fôrderungs Fonds fur die gewerbliche Wirtschaft)
in national and international funding activities including technology policy, EUREKA and the
governmental "Innovation and Technology Fund".

**41**

##### **1996 FIVE-YEAR ASSESSMENT** **INDUSTRIAL MATERIALS AND TECHNOLOGIES**

**43**

**I.** **EXECUTIVE SUMMARY**

**1.0** **Evolution of the Industrial and Materials Technology Programmes**

Among European decision-makers, there is agreement that industrial and materials technologies are of key
importance to the future success of European manufacturing industry. The present IMT programme,
alternatively known as 'Brite-Euram', stems from two RTD programmes, 'Brite' and 'Euram', conducted
under the First Framework Programme. Brite was concerned with a wide range of industrial
manufacturing technologies, while Euram was devoted to advanced materials. In the Second Framework
Programme(FP2), the two subject areas were brought together in 'Brite-Euram', and work related to
aeronautics was added. This arrangement has continued, with the further addition of surface transport
technologies in the current Fourth Framework Programme.

The central emphasis of the work under these programmes has been on precompetitive, generic research,
conducted in a collaborative manner across national borders and involving industry, research institutions
and universities. Throughout, an aim has been to strengthen the scientific and technological basis of
European industry and encourage it to become more competitive at international level. Particular attention
has been paid to the support and encouragement of SMEs, and to the development of European economic
and social cohesion.

Successive IMT programmes have been expanded in funding and scope. FP4 IMT is considerably larger
than FP3, and has a broader range of objectives including improving the quality of life while promoting
sustainable economic growth. IMT funding is summarised below.

FP1 FP2 FP3 FP4

220 MECU 500 MECU 748 MECU 1617 MECU

(Brite + Euram) (Brite-Euram) (Brite-Euram II) (Brite-Euram **III)**

**2.0** **Scope of this** **Five-Year** **Assessment**

This Five-Year Assessment covers the Industrial and Materials Programmes carried out under the Third
Framework Programme( 1991-94) and in the first year of implementation of the Fourth
Framework Programme( 1994-98)

In accordance with the Terms of Reference, the assessment evaluates three principal aspects of the
programmes, namely; relevance of the objectives: the efficiency with which the programmes have been
conducted; and the effectiveness of the programmes. On the basis of these evaluations, recommendations
are made regarding future programmes

This Executive Summary forms the first part of 'Document 1', which also contains the Commission
responses to the Panel's recommendations. The Panel's full Report is presented in 'Document 2'.

This assessment incorporates the final evaluation under the Third Framework Programme.
At the same time, however, the Panel wishes to draw attention to the two Post-Evaluation Reports
covering Areas 1&2 (EUR 17457) and Area 3 (EUR 16948) of this programme. These Post-Evaluations
were carried out in depth, by individuals knowledgeable and experienced in the fields concerned, and
produced a range of specific Conclusions and Recommendations which call for attention. In taking its own
overview, the present Panel aims to add a useful general perspective but this should not be regarded as
repacing or detracting from the important content of the Post-Evaluation Reports.

**3.0** **The Panel's Comments and Findings**

3.1 Relevance of **Obj** ectives

A wide range of **objectives** has been set for the IMT programmes; wider than for typical industrial or
nationally-run programmes. The Panel accepts the **general validity and continuing relevance** of the

**45**

**4.**

objectives, but recognises the difficulty of satisfying all of them simultaneously at project level. In view of
the quickening pace of globalisation in trade, services, and manufacturing, and the need to strengthen
European cohesion, optimal operation of the programme requires prioritisation of objectives, and hence of
project selection criteria, to avoid resources being spread too thinly and focus reduced.

A more suitable format for the IMT programme in FP5 would be a combination of 'horizontal' generic
work with a number of selected areas where a more focussed 'vertical' approach is pursued. These focus
areas might relate to the Task Force interests, though others should also be considered if appropriate. The
modalities of this programme concept, including the possible implementation of additional RTD
instruments, need investigation.

**Selection criteria** such as scientific and technical excellence, economic and social cohesion, sustainable
development and quality of life, generic and precompetitive research, transnational participation, effective
participation of SMEs, and exploitation capability have been generally adhered to.

3.2 Assessment of Efficiency

The issue of **cost-effectiveness** as a measure of the efficiency with which resources have been used in
pursuit of the programme objectives, relates to both Commission resources and programme participants'

resources.

The financial contributions made by the Commission have been generally cost-effective. The funding rules
have provided an appropriate incentive for participation, while at the same time requiring sufficient
investment by participants to ensure serious commitment.

Cost-effectiveness regarding participants' resources depends on the balance between the advantages of
working collaboratively, with a Commission funding contribution, and the extra management effort
needed. Generally this balance is positive, but preparation of proposals requires considerable effort,
averaging about 5% of project cost for each proposal. There is also a high proportion of unselected
proposals, although indications are that in nearly half of these the research would be done anyway. To
reduce the total resource cost of proposals, and to avoid deterring capable potential participants from
applying, the aim should be to improve the success rate to more than 50%. A two-stage procedure, in
which unattractive proposals are rejected on the basis of a less-detailed first-stage submission, should be
considered.

The **efficiency of programme management** relates to activities in advance of the start of project work as
well as to management of the work as it progresses. A continuing concern, expressed both by programme
participants and previous Independent Panels, is the long timescale between close of call for proposals and
start of work. For both FP3 and FP4 IMT programmes, this ranged between 9 and 15 months. This
compares poorly with nationally-run programmes, including those of important competitor nations such as
the USA. This timescale should be reduced to not more than 6 months. Mechanisms to allow work to be

started with limited contract cover prior to final contract signing should be investigated.

Management and monitoring of the work under contract has been generally satisfactory, both by the
Commission and within the project consortia. Positive comments have been received regarding the quality,
competence and dedication of the Commission Scientific Officers. However, there is evidence that, with
the expansion of the IMT programme, the greater number of Member States and the increased complexity
of many proposals, Commission staff are under increasing strain. Programme efficiency is being
endangered and a review of organisation and procedures is needed.

The use of external experts to advise and assist Commission staff can benefit the efficiency and
effectiveness of project management - for example, by providing an independent view, and a healthy
stimulus, at critical project reviews. However, the Panel wishes to emphasise that for efficiency and
continuity reasons it is essential that an adequate core of technically competent personnel should be
maintained within the Commission.

46

**4.**

The Programme Committees, forming the interface at programme level between the Commission and
Member States, have played a key role in the planning, formulation and implementation processes. When
specific funds are allocated to a particular area, eg aeronautics, it is appropriate to involve a panel of
individuals with specialist knowledge to ensure that the programme continues to develop according to
market needs.

The Task Forces have so far had little influence on the IMT Programme. They did not come into existence
until March 1995, by Which time FP3 was in its closing stages, and FP4 had been formulated and the first
call for proposals launched.

In regard to **external monitoring,** the well-established process of 'Post-Evaluation' provides an overall,
in-depth perspective on the whole course of a programme, based on documentation, interviews, visits and
questionnaires. This leads to valuable comments and recommendations for the future, and should be
continued.

A useful complement to Post-Evaluation is the newly instituted 'Annual External Monitoring', which
provides an annual check on progress, thus forming a basis for deciding on adjustments of emphasis,
procedures, etc. However, the the value of the process would be enhanced if that panel had some access to
the views of those involved, perhaps by means of a one-day 'workshop' with national representatives and
project participants.

In relation to **modes of action and tools,** while there is certainly an important continuing role for
precompetitive. generic research, the Panel believes that in view of the increasing intensity of global
competition, shortening of time-to-market, etc, the Commission should now consider complementing this
with nearer-to-market RTD but still stopping short of product development. In addition, consideration
should also be given to RTD invoking Treaty Articles 130 K.L and M and, in selected cases, adding a
proportion of 'directed' research, strategically planned by or on behalf of the Commission, as a
complement to the established style of 'invited' research.

Basic Research projects of industrial relevance, involving research institutions and universities working in
conjuction with industrial enterprises, have proved to be a very valuable aspect of RTD under the IMT
programme and should be continued.

The significance of SMEs within the European industrial scene is well recognised by the Commission and
is fully endorsed by the Panel. In the IMT programmes, **support and encouragement of** SMEs has been
shown by the extensive involvement of already technologically-capable SMEs in industrial RTD projects,
and by the introduction and development of the CRAFT scheme which supports 'cooperative' research
commissioned by groups of SMEs which currently lack an internal RTD capablity.

The participation of technologically-capable SMEs in industrial RTD projects has increased in recent
years. The involvement with larger companies and with academia across national boundaries has provided
SMEs with access to new technology options and new commercial opportunities. On the other hand, there
is evidence that some SMEs are deterred from becoming involved in project proposals because of the long
timescales and low success rate, or because they fear a possible restriction of their flexibility during the
course of a project. SMEs are often subject to more dynamic, fast-changing circumstances than large
organisations.

The CRAFT scheme has proved to be an excellent means for assisting SMEs with a relatively low
technology base, to raise their technical capabilities. The two-step evaluation process is very successful.
The Panel regards CRAFT as a high-priority action, and welcomes the higher funding allocation in FP4.
The CRAFT action is however demanding in terms of Commission management effort, and more attention
to the staffing requirement is needed for the future development of CRAFT. Consideration should be given
to a degree of decentralisation of stimulation to Member States, and the 'bottom-up' approach should be
maintained.

**47**

**4.**

The Panel also wishes to emphasise that the interests and needs of SMEs need to be kept in balance with
those of larger industrial firms, as both large firms and SMEs have important.contributions to make to the
EU.

In regard to Coordination Mechanisms, both the 'Thematic Networks' and the Cluster Concepts' of FP4
are still in a relatively early state. For fuller exploitation of their potential it will be necessary to go
beyond merely the coordination of RTD items which arise separately and independently in various
Commission programme areas. A more proactive, strategic approach will be needed, to set up integrated,
multidisciplinary groups of projects with forward planning to ensure coverage of all aspects essential for
commercial exploitation in a competitive timescale.

3.3 Assessment **of Effectiveness**

The Post-Evaluation Reports for FP3 IMT indicate that the **initial objectives** of promoting technical
research in all Member States, supporting and encouraging SMEs and the collaboration of industry,
research institutions and universities across national boundaries, were met to a high degree. The Reports
also judge positively the achievement of project and programme objectives, including contributing to
European cohesion and economic development. The number of patents filed averaged less than one per
project, but patents are not a universal measure of achievement. Much important innovation within IMT is
not patentable, eg advances in scientific disciplines such as structural analysis, fluid mechanics, etc. The
considerably larger FP4 Programme is still in its early stages and therefore difficult to judge, but strong
responses in most programme areas are apparent.

The **overall objectives** of the IMT Programmes during the past five years have been met in regard to
scientific and technical accomplishments. An open question is whether the resulting contributions have
been adequate to improve European competitiveness, in view of the increasingly strong RTD activities in
competitor countries.

The strengthening of the scientific and technological bases has benefitted SMEs, large companies,
universities and research institutions throughout the EU. In addition to the accumulation of research
results, it contributes to the forming of collaborative relationships after project completion and, especially
for SME participants, to the building of confidence for operating at higher technological levels. In this
respect, participation in the CRAFT scheme has brought particular benefits to SMEs lacking an internal
RTD capability.

It is possible to identify considerable contributions to relevant Community policies through the support of
cooperation across national borders, and the collaboration of SMEs, research institutions and universities.
Economic and social cohesion has been strengthened. There has been a small direct contribution to
employment, by recruitment of individuals to work on projects, and a potential created for larger
employment benefits through industrial exploitation in future. Also, significant scientific/technical results
have been obtained in areas relating to the environment (recycling, less wasteful manufacture, more
environmentally-friendly materials, aircraft noise and exhaust emissions,etc)

In the areas of dissemination and exploitation, a balance needs to be achieved between commercial
exploitation solely by the project partners who have invested in the work, and the distribution of results for
wider benefits to the EU. FP3 IMT results have been given controlled dissemination through numerous
Information Days, Workshops and Conferences. Despite these significant efforts, however, there remains
a concern that many potential European beneficiaries of the results of the IMT programmes are not yet
being reached, and more attention needs to be paid by the Commission to dissemination and exploitation.

Quantitative estimates of exploitation potential in reports produced under Commission contract have
indicated very high economic returns on programme investments. Comparisons should be made between
such estimates and the actual benefits achieved, by re-visiting a sample of projects 3 to 5 years after
project completion.

**48**

**4.**

**3.4** **Importance of the IMT Programme**

The IMT programme is an important and excellent programme. Although the IMT budget in FP4 is
considerably larger than for FP3, it still represents a small proportion of the total requirement for RTD
expenditure in this field in Europe. The high application rate of good-quality industrial research projects
demonstrates the strong willingness of European industry and the research community to participate, and
points to the need for IMT funding to be further expanded to enable the Commission to respond
adequately to this willingness, and to the challenge of strengthening the technological competitiveness of
European industry.

**4.0** **Recommendations**

1. The IMT Programme is considered to be an important and excellent programme. It should be
significantly enhanced within the Framework Programme and very high importance should be allocated
to the objective of strengthening the technological competitiveness of European industries. The
financial means and personnel available for the IMT programme are modest for the task in hand and
should be substantially expanded.

2. The IMT programme should in future combine a basis of 'horizontal' generic work with a number of
selected areas where a more focussed 'vertical' approach is pursued.

3. Measures should be taken to increase the success rate of proposals to higher than 50%. The present low
success rate is considered to discourage prospective participants from submitting proposals potentially
important to future European competitiveness.

4. In order to reduce the cost and staff effort associated with proposal preparation, and the length of the
processing steps, a two-stage submission procedure is recommended.

5. To make the IMT programme more efficient and effective, the timescale between proposal submission
and the start of funding of the work should not exceed 6 months.

6. SMEs qualified to participate in IMT programmes should be encouraged to apply. Better
information and communication with SMEs through expanded networks and other means is necessary.

7. CRAFT is an excellent action in IMT and should receive continued emphasis. Commission staffing
levels in CRAFT should be increased. Partial decentralisation to Member States of the stimulation of

CRAFT involvement should also be considered.

8. Better measures for dissemination of results of IMT projects should be introduced, together with an
increased focus on the encouragement of exploitation.

9. Sample groups of IMT projects should be re-evaluated, 3 to 5 years after completion, to compare the
economic returns actually achieved with the estimates of exploitation potential carried out earlier.

10.To encourage the beneficial spin-off to other industrial sectors of IMT project results aimed firstly at
aeronautical applications, it is recommended that a special information/advisory unit be set up to
promote wider awareness of such advances.

11.Continued emphasis on the use of universities and research institution expertise, especially in Basic
Research, is recommended. Basic Research in support of Industrial technologies is considered very
important in the context of the IMT programme.

49

**(4)**

**4.**

12.The Commission should give consideration, in conjunction with Member States, industry and the
research community, to the introduction on a selective basis of a wider range of instruments for the
conduct of IMT RTD activities.

13.For projects where it is agreed between proposers and the Commission that patents are arelevant
measure of innovation, the gaining of patents should be included in the objectives.

14.The Panel commends the two Post-Evaluation Reports (EUR 17457 and EUR 16948), and the 1995
External Monitoring Report for consideration.

**50**

**II.** **COMMISSION'S** **RESPONSE**

The Commision is grateful to the Five Year Assessment Panel for their thorough and detailed work
in examining activities under the IMT programmes during 1991-1995. The Report is welcomed and
considered fair and balanced. The Panel's positive comments concerning the current programme are
acknowledged. Continuous improvements are being enacted concerning efficiency of management
through modern informatics and streamlining of project reporting, financial control and external
technical assistance. These actions will meet the concerns expressed over limited resources both
within the CRAFT activity and for the management of conventional RTD consortium projects.

The efficient management of the programme requires not only a full transparency of procedures and
commitment of public money, but also the achievement of realistic timescales. The Panel have
rightly identified these as important. Proposal evaluation procedures are under constant review. To
achieve gains in time will essentially require accelerating the consultative and administrative
procedures following proposal evaluation and recommendation, both within the Commission and
with Member States through the Regulatory Committee. The Panel has identified a recognised need
to increase SME participation. This is being continuously encouraged and any further move towards
a more effective decentralized network for stimulating SMEs will ensure a high level of "European
dimension".

As projects under the IMT programme and its predecessors mature, exploitation measures become a
key part of the effectiveness of the programme. The increased attention through the Accompanying
Measures (and under Activity 3 of Framework IV) is ensuring a significant increase in the visibility
of exploitation.

In this context, the "Action Plan for Innovation in Europe" (COM (96) 589) has important messages
for the take-up of results. Transfers of intellectual property, know-how and experience between
different industrial sectors are essential. Patents and licensing, additional financial intstruments, the
effective promotion of technology transfer deals and the encouragement of SME opportunities
within the broader European context will all become crucial as the millenium is approached.

The strengths of programmes such as IMT lie in the achievement of appropriate balances between
basic and applied research on the one hand, and on the other between a generic technical content
together with selected vertical concentrations on strategic themes. The constant accumulation of
experience through project selection and management, coupled with feedback from project
indicators and reviews such as the one just completed by this Five Year Assessment Panel, should
ensure a continued valuable contribution to Europe's industrial RTD base in the future.

**51**

**III.** **EVALUATION PANEL**

**Professor Colin Seabrook**

After an extensive career in engineering and the recipient of a number of awards, Professor Seabrook is
currently visiting professor in Engineering Design at the University of Bath. He continues to serve on a
number of U.K. and Institution of Mechanical Engineering Committees and has provided important
contributions of Advisory Boards within the Department of Trade and Industry and elsewhere within the
U.K. He was formerly Group Research & Development Director - J. H. Fermer & Co. Ltd. Presently a
director of CSMA Ltd., he also runs an engineering consultancy.

**Mr. Frits Klostermann**

Trained in physics and mechanical engineering, Mr. Klostermann spent most of the last 32 years of his
professional career at Philips Research Laboratories, Eindhoven, eventually responsible before his retirement
for the policy of Philips International Research towards the emerging European cooperative R&D
programmes. He currently runs a small consultancy company specialising in electronic device physics topics,
management studies and research evaluation.

**Professor Luis Oro**

Currently professor in chemistry at the University of Zaragoza, Professor Oro has strong links with a broad
range of R&D topics through his recent position as Secretary General of the national plan for Scientific
Research and Technological Development in Spain. He has wide experience of OECD Scientific Policy and
of SME programmes and requirements. Professor Oro is Vice-President of the European Science Foundation.
His research interests include organometallic chemistry and homogenous catalysis. He has published
extensively in the field.

**Dr. Jacques Lukasik**
As scientific director of the Lafarge Group (a major construction company operating in 45 countries), Dr.
Lukasik has considerable experience in the assessment of research projects and interactions with the wider
scientific and technical community outside his own company. His previous experience includes Research
Directorships within CNRS, in particular with International and Industrial relations. He has held a series of
visiting Professorships in the USA, Brazil and Egypt. His technical background in research was originally in
fundamental atomic and molecular physics, quantum optics, optoelectronics and lasers.

Dr. **Hans Joergen Pedersen**
Currently Vice-President for Corporate Technology and Research at Danfoss in Nordborg, Denmark, Dr.
Pedersen was a member of the Industrial and Materials Technologies programme review panel for BriteEuram II (1991-1994). His presence on the IMT Five Year Assessment Panel provided continuity with this
earlier review. Dr. Pedersen has had extensive research management experience in his role as chairman of
the National Committee for the Danish Materials Science Programme, and as a national representative on the
European Organisation for Testing and Certification (EOTC). Dr. Pedersen's particular research interests are
in surface coating technology, microsystems and Total Quality Management.

**Dr.** **Ing.** **Heinrich Bergmann**

Formerly Director of the Institute for Structural Mechanics at the DLR (German Aerospace Research
Establishment), Dr. Bergmann has combined his lifetime's experience with aerospace materials and structures
and general aeronautical engineering with extensive committee advice for National and International R&D
programmes. He has acted as a consultant to the European Space Agency (ESA) on the HERMES Safety
Advisory Committee. Dr. Bergmann is Professor of Mechanics at the University of Hannover, visiting
Professor of Aeronautics at California Institute of Technology and visiting Professor of Engineering
Mechanics at Virginia Polytech, USA.

**52**

**4.**

**Mr. Frank Armstrong** (Rapporteur)
Mr. Armstrong, formerly Director Aerospace Vehicles RAE Farnborough and now a consultant, has held
senior posts in the U.K. Government Aeronautical Research Establishments which involved formulating,
managing and coordinating National Research and Demonstrator programmes in air vehicle technology. He
was a member of the review panel for the 1990-1994 IMT programme Area 3 "Aeronautics" and served as
rapporteur for the IMT Five Year Assessment Panel.

**53**

##### **1996 FIVE-YEAR ASSESSMENT** **STANDARDS» MEASUREMENTS AND TESTING**

**55**

**L** **EXECUTIVE SUMMARY**

The specific programmes for standards, measurement and testing in the Second, Third and Fourth EU
Framework Programmes (FWPs) have undergone major changes. The BCR programme in the second
FWP was a closed, basically subscription-type programme that became the Measurement and Testing
(M&T) programme with open calls on specified themes in the third FWP. Finally the Standards,
Measurement and Testing (SMT) programme in the fourth FWP is a truly open call programme with
broad themes and a small part covering dedicated calls for urgent needs of the European standardisation
bodies and the Commission.

The common objectives of these three special programmes are to strengthen the functioning of the
European internal market and the competitiveness of the European industry and also to meet the needs of
society. The themes of the programmes have been enlarged from fabrication of Certified Reference
Materials (CRMs) and measurement support to Community Policies in the BCR, to measurements for
Quality European Products, research related to written standards and technical support to trade as well as
measurements related to the needs of society. This larger scope is also reflected in the budgets that have
increased from 59.2 MECU in the BCR to 184 MECU in the SMT programme. The funds for the Joint
Research Centre (JRC) are not included in these figures as the JRC is not included in this evaluation.

**The European added value is very high in the BCR, M&T and SMT programmes as they address**
**areas that are essential for the EU to function.** The importance of this has grown continuously and is
today much higher than at the time of the BCR programme. Today the directives and standards that are
developed are in use and must be enforced. The necessary conformance assessment depends upon
harmonised and traceable measurements and a uniform and reliable measurement and testing
infrastructure. In addition there is a huge backlog in the development of standards. **All of this needs**
**strong support from the specific programmes on standards, measurement and testing.**

The Panel has based its five year evaluation of the BCR/M&T/SMT programmes upon extensive
interviews, earlier evaluations of parts of the programmes and material provided by the programme staff.
The resulting recommendations are such that some of them can be implemented almost immediately by the
programme management, others need decisions by the Commission but could still be implemented during
the present SMT programme and the remaining ones are intended as inputs to the fifth FWP.

The role of the programme staff has evolved considerably during the assessment period. The role of
scientific officers has changed from scientific involvement to a more administrative, consulting and
monitoring role. This has increased efficiency and more projects are now handled per scientific officer
than before. This change has at times been painful and the management has not always been able to
respond to the changing needs. The project teams and members of the Regulatory Committee see the SMT
programme presently as being well run and the staff as helpful and responsive to needs. **The Panel**
**recommends that the management of the programme is developed to a more team-based form with**
**much delegation and involvement of the whole** **staff.**

Problems are seen in the slow and complicated general procedures of the FWP in the Commission for
approving project proposals and letting contracts. The problem is greatest in the dedicated calls where the
time from proposing a topic to the signing of a resulting project takes at least two years. It is felt that the
procedures that are created to ensure impartiality and independence may create a false sense of security.
**The Panel recommends that the time from closing a call to signing the contracts for approved**
**projects shall be no longer than six months** and a special approval process that is no longer than three
months shall be created for small and short projects for solving urgent European needs.

The dissemination and exploitation of results of single projects was not a concern in the BCR programme
as the results, like developed CRMs and intercomparisons, had a clear user group that was easily
informed. In the subsequent programmes this question needs more attention as the results of the projects
can be used widely and are from more diverse areas. Despite only 36 of the 173 projects in the M&T

**57**

programme having been completed, the output of projects is already impressive. The Panel recommends
that the encouraging start to **the dissemination and exploitation of the results of the M&T programme**
**and in the future the SMT programme shall be continued effectively and more systematically and be**
**followed up** so that maximum benefits of the project results are obtained.

The concept of dedicated calls in the SMT programme is seen as very good. The Panel believes it is the
right concept to provide solutions to specific European needs in standardisation, measurement and testing.
**The Panel recommends that the dedicated calls concept is continued and further developed** by
allocating more funds for this purpose, increasing the number of European groupings that propose topics
for these calls and speeding up the procedures for approving project proposals for these calls.

As mentioned earlier, the BCR/M&T/SMT programmes have all had a major significance for the
implementation of EU policy. **The panel sees a strong need, that has been expressed from many**
**sources, to keep and strengthen the SMT in the fifth FWP as a separate programme** or in some other
way organise and secure that it remains a strong unit with its own identity and funding. The overall
objectives of the present SMT programme can be used as the basis for the next programme but some
revision of focus and objectives as well as additional detailed measurable objectives against which the
progress of the programme can measured are recommended.

**The panel recommends that the future programme for standards, measurement and testing shall**
**include dedicated, special focused and open calls.** With this combination, all types of European needs in
this field can be taken care of. The allocation of funds to different calls and themes should allow for some

later adjustment, for instance by the Regulatory Committee, based upon the experience of finished calls
and emerging new needs. Because of the continuing substantial needs in the fields of the **programmes it is**
**recommended that the total financial resources for the future programme are raised to 300** MECU
without taking into account the part of the budget allocated to the JRC. The relative share of funds for the
standards, measurement and testing programme in the FWP V should be clearly higher than at present.

The production of CRMs has been a valuable contribution to European metrology in the programmes
evaluated. **The panel sees that these activities also need support in the future programme.** This should
be organised within the suggested framework in such a way that the important European needs in the field
are fulfilled and the high quality of CRMs is secured and maintained.

The Panel had to do this evaluation with a very tight schedule of only three months. The help of the SMT
programme staff was very valuable and without it the work could not have been completed according to
the given schedule. The panel members have all actively contributed to the report and the conclusions and
recommendations of the panel are unanimous.

**58**

**II.** **COMMISSION'S RESPONSE**

The Commission is very grateful to the Five Year Assessment Panel for its thorough and detailed work in
examining the activities of the Standards, Measurements and Testing (SMT) programme during the period
1991-1995. The constructive recommendations made by the panel are welcomed and will be taken into
account during the 4th Framework Programme as well as during the planning of activities related to
standards, measurements and testing in the 5th Framework Programme.

As pointed out by the panel, the SMT programme has undergone fundamental changes in its scope and the
way that it operates during the last five years. These changes were necessary in order to ensure that the
programme is managed in the same manner as other specific industrial related research programmes in the
4th Framework Programme. This evolution is presently continuing towards a form using more delegation
and team work.

With respect to other recommendations to be implemented in the short term, the Commission fully
recognizes the importance of ensuring that calls for proposals are not over subscribed, of reducing delays
in the project evaluation and selection process, and of ensuring the exploitation and the dissemination of
the results of projects. Appropriate measures are being taken, and will continue to be taken, in these
directions. One important development, that will help to increase the exploitation and the dissemination of
results, is that the progress of projects is now being monitoring against clear quality indicators. The
Commission's preliminary guidelines for the 5th Framework Programme (Commission documents
COM(96)332 and COM (96)595), echo the panel's, recommendations concerning the need to focus
Community research efforts, to increase the flexibility of Community research programmes in order to
respond to needs as they arise, and to reduce the time taken for project evaluation and selection.

It is particularly pleasing that the panel has recognized the merits of the dedicated call system in the SMT
programme. The system, which was largely experimental when it was first set-up, has already evolved
considerably and will continue to do so in the 4th Framework Programme. It allows Community RTD
activities linked to standardization to be precisely and efficiently focused where the real urgent needs are,
and may provide a model for similar systems to be set up in the 5th Framework Programme.

The Commission fully agrees with the panel on the paramount importance of standards, measurements and
testing for the future competitiveness of European industry in the global economy and for the general wellbeing of society.

59

**III.** **EVALUATION PANEL**

**Professor Freddy ADAMS (Rapporteur)**
He is currently professor of chemistry at the University of Antwerp where he was until October 1995
rector and chairman of the University Research Council. He is member of a number of Belgium Research
Institutions. His main research interest focus on environmental chemistry and material science.

**Mr. Michael BROCK**

Trained in electrotechnic and acoustic engineering, Mr. Brock is Director of the Transducer Products
Division at Bruël and Kjaer. In addition, he acts as consultant in management and strategical planning of
technological complex companies.

**Mr. Pierre CROON**

Mr. Croon is currently General Director of the Belgium Institute for Standardization. He is participating
actively in the European Committee for Standardization (CEN-CENELEC).

**Mr.** **Gôran** **LINDHOLM (Chairman)**
Mr. Lindholm is currently Director of NORDTEST, organization for testing in the Nordic countries. His
previous experience includes directorships at international level in the electronic and information
technologies area.

**Dr. Walter RAULS**

Currently managing Director of the Standards Commission for testing materials in DIN (German Institut
for Standardization). He is also participating actively in various CEN working groups.

**Mr. Michel ROUSSEAU**

Trained in mechanical engineering, Mr. Rousseau is auditor for the French Committee for Accreditation
(COFRAC).

**Dr. James TWEED**
Dr. Tweed is currently technical manager for programmes on testing and design methods at AEA
Technology Harwell (UK). His technical background in research is in polymers and metal composites.

60

##### **1996 FIVE-YEAR ASSESSMENT** **ENVIRONMENT AND CLIMATE**

**61**

**I.** **EXECUTIVE SUMMARY**

**1.0** **Introduction**

**The Five Year Assessment Exercise**

This is the **Executive Summary** of the 1996 Five Year Assessment of Specific Programmes in the Fields
of Environment and Climate Research and Technological Development (RTD). It covers all Framework
research activities supported by the Commission in these areas over the period 1991 to mid-1996,
including activities supported by the Third Framework Programme (1990-94), and all activities included to
date in the Fourth Framework Programme (1994-98). In compliance with COM (96) 220 final (22 May
1996), this Five Year Assessment constitutes the final evaluation of the programme launched in the field of
Environment under the Third Framework Programme.

The Assessment Panel charged with conducting the Five Year Assessment met on several occasions to
discuss the background documentation provided by the Commission services; to interview Commission
staff and representatives of organisations with an interest in environmental research; and to draft the
Assessment Report. Individual Panel Members held separate discussions with senior members of the
scientific community across Europe. A questionnaire was also sent by Technopolis Ltd. to all project
coordinators of the Environment Programme (1990-94) and the Environment and Climate Programme
(1994-98). In total, 805 questionnaires were distributed and 402 coordinators responded by the cut-off
date of mid-September (50% response rate).

**Environment and Climate Programmes**
The Specific Programme in the Field of Environment RTD (1990-94) was aimed at contributing to the
scientific and technical basis for the implementation of EU policy. It was divided into the following areas

Participation in Global Change Programmes
Technologies and Engineering for The Environment
Research on Economic and Social Aspects of Environmental Issues
Technical and Natural Risks

Some 554 projects were funded over the course of the programme, which had a project budget of 300
MECU and an overall programme budget of 315 MECU.

The Specific Programme in the Field of Environment and Climate RTD (1994-98) has aims similar to its
predecessor. Amongst other things, it is intended to

Contribute to world programmes of research into global change
Satisfy the demands of subsidiarity and contribute to cohesion via the promotion of scientific and
technological cooperation and integration between European universities, research institutes and
industry
Act in concert with the activities of the Member States

Be of a high quality and strengthen the scientific base in Europe
Help develop the scientific knowledge and technical competence needed to fulfill environmental
policy mandates
Make a significant contribution to growth, competitiveness and employment and encourage the
participation of SMEs

The programme has a budget of 567 MECU and is subdivided into four main work areas

Natural Environment, Environmental Quality and Global Change (47% of the budget)
Environmental Technologies (25%)
Space Techniques Applied to Environmental Monitoring and Research (20.5%)
Human Dimensions of Environmental Change (7.5%)

63

**6.**

By mid-1996, 313 project awards calling for an EU Contribution of 246 MECU had been granted.
Another 22 Concerted Actions (EU Contribution of 5.3 MECU) had also been ratified.

Higher Education Institutes (HEIs) and Research and Technology Organisations (RTOs) dominate the
programme. For Shared Cost Actions, there were 2,125 successful institutions. These include 1,040
HEIs; 863 RTOs; 123 firms; and 99 other organisations. The low level of industrial participation is not
worthy.

**2.0** **The Findings of the Assessment Panel**

The Specific Programmes of Research and Technological Development (RTD) in the fields of
Environment (1990-94) and Environment and Climate (1994-98) have made important contributions to the
development of environmental science. Programme goals were highly relevant to environmental science
agendas both in Europe and globally, and the work conducted had the potential to contribute to the
evolution of environmental policy. Programme execution was for the most part satisfactory, and goal
attainment in terms of scientific achievement was sound in most areas and impressive in others. There
was more limited impact on policy development, however, and low industrial participation meant that
contributions to European competitiveness were marginal.

**Relevance**

The need for a programme of RTD activities in the field of Environment and Climate remains. An
increasing emphasis on sustainable development will be best served by a highly visible programme of
support for the European environmental research community which past and present research programmes
have helped to create.

The centre piece of this support should be a specific framework programme which combines the current
Environment and Climate and MAST programmes. Efficient mechanisms are needed to link this with the
environmental research needs and activities of the Directorate General of the Commission responsible for
the Environment (DGXI), and with the research needs of other relevant Directorates General. It should
also be linked with the work of other bodies and agencies such as the European Environment Agency
(EEA), the European Space Agency (ESA) and the European Meteorological Satellite Organisation
(EUMETSAT). Improved links with the Joint Research Centre's Environment Institute are needed to
make the work of this JRC institute more visible to the European environmental science community.

The goals of the programme are also still appropriate. In particular, there is still a need to strengthen and
maintain the European scientific base and to conduct policy-relevant research. Ensuring the participation
of industry is also important to encourage the spread of environmental best-practice as this becomes an
increasingly important factor in competitiveness.

Programme compositions were largely in line with programme goals. Environment and Climate projects
comprised a mix of fundamental and applied research which was relevant to enhanced understanding and,
to a more limited extent, to policy development. Participants saw projects as a way of expanding
existingknow-how, creating new European networks and developing new tools and techniques via access
to complementary sources of expertise in other European organisations. The majority of projects,
however, were not geared towards enhancing industrial competitiveness.

**Efficiency**
Work within the environment programmes was for the most part adequately financed and conducted in an
efficient manner. The programme appears to have been conducted in a cost-effective manner.

Management and administration of the programme was for the most part sound. Some problematic
management issues should be resolved with the appointment of a new Programme Director; general
administration was satisfactory; and conduct at the project officer level was commendable. Participants
were appreciative of the help given by project officials at all stages of the programme life-cycle. Most felt

64

**6.**

that Commission procedures for making applications were clear and easy to follow, though many still
regarded them as too slow. Late payment was also a common complaint across the research community.

There were serious problems in terms of the relationship between the Commission services and the
Programme Management Committee (COPEC). The transition from an advisory body to a management
committee has not been smooth or well managed, and efforts will be needed in the future to establish
productive working relations.

A number of other administrative problems were also apparent. In isolation none appeared particularly
severe, but a timely review of administrative functions, procedures and resources, especially staffing
policies and practices, would help avert problems and maintain current performance levels.

The range of support modes open to the Commission services (Shared Cost Actions, Concerted Actions
and Accompanying Measures) was satisfactory, though some aspects of their use were not. Lack of
support for projects involving large numbers of institutions was regretted. Although these have high
coordination and communication costs, the benefits of participation often outweigh the costs and a flexible
approach to their inclusion is recommended. There was also a marked tendency to reduce project funding
levels with no corresponding reduction in project objectives or ambition. This should be avoided. New
transparent procedures for assessing project proposals were welcomed, however.

Effectiveness

There is little doubt that participants were satisfied with project achievements. Immediate goal realisation
occurred in terms of networking and financial security, whereas knowledge enhancement was seen as a
medium-term outcome.

Project coordinators were convinced that projects had realised their scientific potential and that the
European environmental science base had been strengthened as a consequence. Networking had helped
forge a dynamic European research community which was well integrated with national and global
research agendas and associated communities.

Some research results had fed into policy discussions, though overall policy impact had been limited. This
is largely a function of the nature of the work conducted and the complex, lengthy and non-linear ways in
which scientific evidence feeds into and affects policy environments and decisions. Lack of policy impact
had been exacerbated, however, by weak links between the DGXII (Science, Research and Development)
research community and policy-making bodies such as DGXI (Environment) and other Digs concerned
with environmental issues.

The networking aspects of the programme had contributed greatly to the cohesion aspects of EU policy,
though the Less Favoured Regions still won a disproportionately small share of projects.

The low level of industrial participation in the programme meant that overall contributions to
competitiveness had been small. In future greater efforts will be needed to encourage industrialists to join
projects and networks and to realise the industrial potential of projects.

The overlap between the areas of interest of the Specific Programmes and national programmes is
sufficient to ensure that the Specific Programmes remain relevant to national needs, but not so extensive
that they can be considered a substitute for national efforts.

All the available evidence points to genuine additionality and subsidiarity, with participation in the EU
Specific Programmes adding an international dimension via access to complementary assets not attainable
within national boundaries. Many of the projects concerned with global and regional issues could not have
been undertaken without Commission support.

Dissemination of the results of project outputs to other participants in the programmes and to the scientific
community at large had taken place. Dissemination to policy-making 'end-users' and to the industrial

65

**(5)**

community had been more disappointing. New mechanisms to ensure this occurs need to be put in place.
This could involve the use of external consultants to advise on dissemination strategies and restructuring
some future activities into User-focused areas defined in part by relevant user communities.

**3.0** **Recommendations of the Assessment Panel**

Continuation

_The Specific Programme of_ _**RTD**_ _in the field of Environment and Climate should continue into the Fifth_
_Framework Programme._ Dispersing environment-related RTD activities into other programmes is not a
viable or desirable option. A critical mass of highly visible environment-related research is needed if
sustainable development is to be a keynote feature of the Fifth Framework Programme. This does not
mean, however, that all environment-related RTD activities should be centralised in DGXII(Science,
Research and Development). Other Directorates General, especially DGXI (Environment), DGVI
(Agriculture), DGVII (Transport) and DGXVII (Energy) should be encouraged to continue and expand
their environment-related RTD activities, with improved mechanisms in place to ensure synergy rather
than duplication.

Integration

_The Specific Programme of RTD in Marine Science and Technology (MAST) should be integrated into_
_an expanded Environment and Climate RTD programme._ The arguments in favour of integration out
weigh those for its continued, separate existence. In particular, there is a need to strengthen the relative
position of environment-related RTD in order to ensure that sustainable development is central to the Fifth
Framework programme, and this can best be done by having a critical mass of flagship RTD in a highly
visible Specific Programme.

Networking

_The emphasis on networking within the Specific Programme of RTD in the field of Environment and_
_Climate should be maintained and strengthened._ The benefits of networking across Europe were highly
prized, readily attained and much appreciated by the environmental research community. These benefits
are likely to be short lived, however, in the absence of sustained efforts to nurture a culture of
cooperation, collaboration and networking. The Panel thus recommends a continued focus on network
building and maintenance between:

Participants within individual projects
Participants in different projects
All interested parties in specific topic areas, especially those requiring interdisciplinary approaches

Particular efforts should be made to include industry in Environment and Climate networks and projects.

Segmentation

_The programme structure should segment and differentiate between User-focused and other types of_
_research._ Introducing policy-driven, customer-oriented User-focused areas would focus programme
activities and increase the likelihood of policy relevant work having appropriate policy impacts. Userfocused areas should be limited in number; account for no more than 20% of the overall budget; have
identifiable user communities in environmental and industrial policy-making circles; and a high policy
relevance in the short- to medium-term. All other proposals should be oriented towards the medium- to
longer term. Possible User-focused areas might include activities geared towards

Global issues and concerns, such as uncertainties related to the climate change agenda
The needs of multi-nation regional systems and regional policy makers, such as regional seas

issues
Some of the specific needs.of industry and the industrial policy community, such as research on
cleaner production technologies

**66**

**6.**

Some of the specific needs of DGXI(Environment) related to the implementation of the United
Nations Framework Convention on Climate Change; waste treatment and disposal technologies;
and public perception of environmental issues

**Selection**

_**Selection procedures should reflect the difference between User-focused and other research projects,**_
_**with additional criteria for User-focused projects.**_ In addition to satisfying excellence criteria, proposals
in User-focused areas should specify policy relevance, potential users and ways in which project results
might be exploited in policy development, and this information should be taken into consideration in
selection decisions. Project proposals not targeted at User-focused areas should not be expected to contain
such detailed information on potential exploitation routes, though its presence would be welcomed given
that all projects supported within the Specific Programme should be of some relevance - direct or indirect

- to environmental policy making.

**Shaping**
_**Individual customer communities for specific User-focused areas (e.g. DGXI (Environment), other**_
_**DGs, national policymakers, industry etc.) should be consulted and involved more**_ _extensively_ _**in the**_
_**formulation and specification of research agendas and work programmes.**_ This would focus agendas
more closely on customer needs and would sensitise these user communities to the existence and potential
utility of research outputs. For the remainder of the programme, the Commission Services should define
its scope by specifying broad areas of interest and concern to multiple user communities - including the
scientific community itself.

**Linkage**
_Linkages between_ _**DGXII**_ _(Science,_ _**Research and Development) and DGXI**_ _(Environment) need_ _**to**_ _be_
_improved, particularly_ _**in the policy-relevant**_ _User-focused areas._ Currently DGXI (Environment) is
unable to 'see' the results of work supported by DGXII (Science, Research and Development) or to
appreciate their policy relevance. In part this can be rectified by better 'packaging' of results and
improved communication channels and mechanisms between the two DGs. Critically, however, DGXI has
to show more concern and play a greater part in the determination of the research agenda and work
programme for a specific User-focused area geared towards its short- and medium-term needs as a
customer of policy-relevant research outputs. In particular, **DGXI** should make public its medium-term
work agenda and its implications for research. Similarly, other DGs with an interest in environmental
issues (e.g. Agriculture, Energy and Transport) should be encouraged to play a greater role in the setting
of research agendas within DGXII.

**Dissemination**

_**Dissemination**_ _practices_ _**should be improved via better packaging and targeting and the**_ _customised_
_**delivery of outputs to user communities.**_ Specifying User-focused areas and involving relevant user
communities in agenda setting will undoubtedly improve the prospects for the take-up of policy-relevant
research results. There is still scope, however, for delivery mechanisms to be improved. User
communities are interested in problem-solving, and thus in information which can help them solve
problems - whatever its origin. Policy-relevant results of environmental research supported by DGXII
(Science, Research and Development) therefore have to be packaged together with other data relevant to_
specific problems. This then has to be delivered to well specified target audiences via a number of routes,
e.g. publications, brochures, seminars, workshops etc. DGXII should continue to explore ways of doing
this, possibly using external consultants to define an effective dissemination strategy.

**Industry**
_**An attempt should be made to increase industrial participation via a**_ _sub-programme_ _oriented_ _towards_
_the needs of firms._ The evolution and implementation of technologies with reduced environmental
impacts is crucial to the concept of sustainable development, yet industrial participation is currently low.
Ways of reducing barriers to entry and making participation more attractive should be sought. A Userfocused area oriented towards some of the needs of industry is advocated. The alternative sign-posting of

**67**

industrial applicants to other Specific Programmes, e.g. Industrial and Materials Technologies, is not
recommended.

**COPEC**

_**Attention needs to be paid to the relationship between the Commission services and the Programme**_
_**Committee for the Specific Programme of RTD in the field of Environment and**_ _**Climate(COPEC).**_ The
change in status from an advisory body to a management committee has led to a more formal and strained
working relationship with the Commission services. Action is necessary to restore levels of mutual trust
and cooperation. The emphasis should be on striking a balance between the relative powers of the
Commission services and the representatives of national governments.

**Financing**
_**Pressures to cut individual project funding and spread resources too thinly over**_ _a_ _**larger number of**_
_**projects should be strongly resisted.**_ Project participants generally receive less than the amounts requested
in their proposals, with little corresponding reduction in the scope or scale of project objectives. For
many this involves seeking supplementary funding and an effective reduction of the proportion of the total
cost covered by the Commission. The need to supplement funds suggests that original requests are not
overly inflated, and that the scientific integrity of the work could suffer if these requests are not met fully.
To avoid this happening, the Commission services should scrutinise suggested cuts carefully and resist
pressures to spread resources too thinly. Access to advice from personnel capable of making realistic
estimates of project resource requirements, e.g. those with experience of R&D project management,
should be be considered via the appointment of external 'reference groups'.

**Administrative Procedures and Resources**

_A review_ _**of administrative procedures and resources should be undertaken**_ _across_ _**the**_ _Commission_

_services_ _**dealing with Environment and Climate RTD activities.**_
Complex bureaucratic procedures lie at the heart of complaints about slow application processes and late
payments. The Assessment Panel was also told that expanded workloads were not commensurate with
modest staff increases in the Commission services over the past five years, and that overload problems
were acute in some programme areas. If this is indeed the case, immediate step child be taken to rectify
the situation. Programme management has to date been well regarded by the research community, and it
is important not to let under staffing threaten this situation.

68

**IL** **COMMISSION'S** **RESPONSE**

The Commission welcomes this independent evaluation of EU RTD in environment and climate and will
take the recommendations into account both in implementing the on-going programme under the Fourth
Framework Programme and developing the Fifth Framework Programme for the end of the century.

The Commission accepts that research into environmental issues at the EU level is essential to respond to
societal needs and aspirations, as well as to address fundamental problems arising from the land, sea and
air, to assure a safer and healthier planet within the context of sustainable development and growth. For
this reason, a major theme in the Fifth Framework Programme should be unlocking the resources of the
living world and the ecosystem in which major problems such as our future water, the environment and
health, better protection from catastrophes, eco-management and the urban environment will be tackled.

Networks within and beyond Europe have been an essential element to successful environmental research
and European teams have been at the cutting edge of addressing our global challenges through their
contributions to international initiatives such as the Intergovernmental Panel on Climate Change (IPCC),
the International Geosphere-Biosphere Programme (IGBP), the World Climate Research Programme
(WCRP) and the International Human Dimensions of Global Environmental Change Programme (HDP).
These links will continue to be vital in the search for solutions to issues and problems not found or
concentrated solely in Europe.

The need to involve the widest possible interest groups in the shape and direction of future environmental
research is a key element to securing transparency and meeting the citizens' needs. This will continue to
be a priority for the Commission services in the development and implementation of its research activities.
Improvements have already been made to strengthen links between the operational research Directorates
General and the policy making Directorates General to ensure greater synergy and coherency and this
process will be further developed.

Commission services recognise that continued efforts need to be made to promote the opportunities for
industrial participation - both through large and small to medium-sized enterprises - in the research
activities. These opportunities are being taken up in specific areas of existing research activities such as
environmental technologies and space, but more work remains to be undertaken to ensure better public private sector, and academic-industrial, synergies to maximise the potential opportunities and benefits
from collaborative environmental research. Such research must never be seen as exclusively for public
sector organisations or be carried out only through public financing. Nevertheless, the Commission
considers that the establishment of an industrial subprogramme specifically directed towards the needs of
industry, as proposed by the Panel, is not the best way forward to assure such synergy or industrial
involvement.

Numerous steps are already taken to put EU funded environmental research to the best use. Nevertheless,
further work will be undertaken by the Commission services to ensure greater awareness and visibility for
the output of EU environmental research. Solutions need to be found to ensure that the vast array of
results come in a digestible form as quickly as possible to the attention of those who can best use and need
them.

Some brief findings are made by the panel concerning work undertaken by the Environment Institute of
the Joint Research Centre. Whilst the panel correctly indicates that an assessment of the Environment
Institute was not within the remit of the panel and has been undertaken by a separate Visiting Group under
the chairmanship of Professor Carlos A. Borrego, the main report nevertheless refers to expenditure by
the Environment Institute on environment-related work. These figures are misleading: only 37% of the
core funding of the Institute (which in 1995 totalled 43.5 MECU and not 46 MECU, as set out in the
report) is devoted to environmental research, the remaining 63% is allocated for scientific and technical
support to EU policies developed and implemented by various Commission services (i.e. primarily the
preparation and implementation of EU directives). The amount expended by the Institute on research

69

**6.**

during the period of the Fourth Framework: Programme will, therefore, represent 60-64 MECU and not
half the budget of the whole environment and climate programme (566.5 MECU), as indicated by the
panel.

Finally, the Commission recognises in its documents on preliminary guidelines for the Fifth Framework
Programme that changes need to be made in some of its management techniques and arrangements to
ensure more efficient management.

Issues raised in the evaluation report which concern the management of the research activities will be
taken forward in this context. The Commission services do, however, attach considerable importance to
the need for constructive and open dialogue with the programme committee, whilst preserving the
institutional responsibilities for the various actors in the research process, and steps are already being
initiated to ensure the mutual trust and cooperation essential for the wellbeing of environmental research.

**70**

**IIL** **EVALUATION PANEL**

**Dr. Lea Kauppi** (Chairman)
Director General of the Finnish Environment Institute since 1995

**Prof.** **Niels Busch**

Director of the Danish Russian Energy Research Institute, Moscow
Former Director of RISO National Laboratory (Denmark)

**Prof.** **Eloy** **Garcia** **Calvo** (withdrew due to ill health)
Professor of meteorology of the Universidad de Alcala, Spain
Manager of the Spanish national R&D programme on climate

**Prof.** **Mike Chadwick**

Director (Europe) Leadership pour l'environnement et le développement (LEAD), Switzerland
Former Director of the Stockholm Environment Institute and Chairman of the 1995 Annual Monitoring
Panel for the Environment and Climate RTD Programme

**Prof.** **Wim** **Harder**

Scientific Director at TNO Institute of Environmental Sciences, Energy Research and Process Innovation,
The Netherlands

**Prof.** **André Lebeau**

Chair of the space and space technology programmes, Conservatoire national des arts et métiers, France
Former President of CNES and Director of Méteo France

Former Head of ESA and Scientific Director of CNES

**Prof.** **Francesco Mauro**

Director of the Environmental Department of the National Agency of New Technologies, Energy and
Environment (ENEA), Italy
Member of Italian Delegation for the conventions on biological diversity, and desertification and drought.

**Mr. Ken Guy** (Rapporteur)
Director of Technopolis Limited, an innovation policy consultancy, UK
Former senior fellow at the Science Policy Research Unit (SPRU) at Sussex University.

71

##### **1996 FIVE-YEAR ASSESSMENT** **MARINE SCIENCE AND TECHNOLOGY**

**73**

**I.** **EXECUTIVE SUMMARY**

**BACKGROUND**

Article 4.2 of the Decision 94/804/EC on the specific RTD programme in the field of marine science and
technology stipulates that the Commission shall have an external assessment conducted by independent
experts of the activities carried out within the fields/areas/domains covered by this programme, and of their
management during the five years preceding this assessment.

This 'Five-year Assessment' covers the relevant activities carried out during the Third Framework
Programme (from 1991 to 1994) and those carried out during the first part of implementation of the Fourth
Framework Programme (until early 1996). Where possible, major achievements arising from the Second
Framework Programme (1987 to 1991) are also reported. This assessment subsumes the final evaluation of
the MAST-II specific programme under the Third Framework Programme. It also includes an assessment of
the early stages of M AST-Ill, together with an analysis of the most significant outputs from MAST-I.

The Assessment Panel consisted of six independent experts who considered the Programme's relevance,
efficiency, and effectiveness in delivering its objectives, and sought to highlight examples of major
achievements. The Panel also discussed the requirements for future Community research in marine science
and technology.

The Panel undertook personal interviews with:- selected project coordinators; all national delegations on the
MAST Committee (MAST-COM); representatives from Commission Directorates DGIII, DGXI and
DGXIV; MAST Programme staff; and the Secretary of the EUROMAR Programme. The Panel also
examined the results of a standard questionnaire sent to all project coordinators within MAST, as well as a
wide range of literature relating to the MAST Programme and individual projects.

The Assessment Panel's report has been prepared in two parts. Part 1 (this document) is the Executive
Summary containing the main findings and recommendations arising from the Panel's deliberations. Part 2
(Final Report) describes the development of the MAST Programme; discusses the impact of the Programme
on a wide range of issues; gives examples of major achievements; presents detailed conclusions and
recommendations; and includes various statistics related to the Programme.

**DEVELOPMENT OF THE MAST PROGRAMME**

The MAST Programme has developed through MAST-I (1989 to 1991), MAST-II (1991 to 1994) and
M AST-Ill (1994 to 1998). MAST-I was essentially conceived as a pilot programme (budget 50MECU)
covering a wide field, with a view to developing a successor programme. Under MAST-II, certain activities
begun under MAST-I were developed, new topics were introduced, and the geographical coverage was
expanded. This was summed up in Annex I to the Council Decision of 7 June 1991 (91/351/EEC). MAST-II
had a final budget of 118MECU, and covered five areas:- (I) Basic and Applied Marine Science; (II) Coastal
Zone Science and Engineering; (III) Marine Technology; (IV) Supporting Initiatives; and (V) Targeted
Projects. An area of work included at a late stage was that of Risk Evaluation.

The M AST-Ill Programme was set out in the Council Decision of 23 November 1994 (94/804/EC), and
extends, develops and refocuses the activities carried out under the first two MAST Programmes. Its general
objective is "to foster the scientific knowledge and technological development necessary to understand how
marine systems function at basin scales, in order to prepare for sustainable use of the oceans consistent with
the preservation of marine environmental quality and to determine their role in global change." The budget
for M AST-Ill is 244MECU, and the Programme comprises (A) Marine Science; (B) Strategic Marine
Research; (C) Marine Technology and (D) Supporting Initiatives. In addition, greater efforts are being made
to involve SMEs (Small and Medium Enterprises) in the Programme, and stress is being laid on parallel
initiatives (eg. dissemination/ exploitation/ implementation).

**75**

**CONCLUSIONS**

The main findings which the Panel obtained from their assessment are given below under the general
headings of the effectiveness, efficiency and relevance of the MAST Programme:

Effectiveness of MAST Programme

- The MAST Programme has successfully mobilized a European-wide scientific community in marine
science, through the development of networks associating a large number of laboratories from different
types of organisations and different countries. The Programme has stimulated interdisciplinary research
projects which are necessary to achieve the MAST objectives, and it has increased the capacity of
Europe to participate in large international océanographie programmes.

**•**

**•**

**•**

Major breakthroughs have been achieved on a wide range of scientific and (to a lesser extent)
technological issues. The Panel is convinced that most of these major results could not have been
achieved without the organisational framework and interdisciplinary approach made possible by the
funding opportunities within the MAST Programme. MAST is the only one of the various European and
International programmes which has both the mandate and the budget to encourage European
collaboration in major research projects.

The results of the MAST Programme constitute an important scientific and technological asset, which in
the long term will be used for future technological developments, and for management and policy
initiatives. The strengthened knowledge base developed in MAST therefore has an intrinsic economic
value in the long term.

The development of the code on data management has greatly enhanced the accessibility and value of the
data collected during all projects in the MAST-II and M AST-Ill Programmes.

The MAST Programme objectives are relevant to policies in many sectors, including fisheries,
environment, transportation, industry etc. It is difficult as yet to pinpoint any direct benefits of the
MAST Programme to national or European maritime policies - indeed most Member States do not
possess integrated policies on marine affairs. However MAST has strengthened the basic knowledge and
the scientific expertise required for their development. Within the Commission there seems to be quite
good liaison between the MAST office and the Fisheries and Environment Directorates, but links with
the Industry Directorate and the "Maritime systems of the future" Task Force need to be strengthened.

The MAST Programme seems to have had a significant impact on the coordination between national and
European marine science R&D policies. In many cases national marine science R&D policies have been
based largely on experience gained in implementing the MAST Programme. There seems little doubt
also that the MAST Programme has strengthened cooperation amongst national groups.

The objective of enhancing European competitiveness in marine technology has only been achieved to a
limited extent, and the Panel believes that this is due to at least two factors:- the involvement of
industrial companies in the Programme has been disappointing, and dissemination of the results of
MAST to the industrial community has been unsatisfactory.

The Panel welcomes the moves in MAST-III towards the development of better links between research
workers and the users of their results, whether they be industrial companies or policy makers.

Within the MAST Programme the "Technology stimulation measures for SMEs" have had a rather poor
success rate. It seems that either the availability of this supporting initiative from the Commission was
not disseminated well enough, or that the stimulation measures do not meet the needs of SMEs with
respect to their R&D activities.

76

Efficiency of MAST Programme

**•** [The Panel was disappointed to learn that very few of the members of MAST-COM came from an ]
industrial background, and was also surprised to discover that some members knew little of the MAST
projects in which their national laboratories were involved, or of the main results of the Programme.

- The Panel can confirm that the Commission's Programme Officers in the MAST office are highly

regarded by the MAST community.

- The publication by the MAST office earlier this year of the booklet "Evaluation of Proposals" has been

warmly applauded. However the project authorization procedure is still not clear to most of the MAST
applicants. The Panel also noted that contract negotiation almost always resulted in a budget reduction,
usually without a corresponding reduction in the project's objectives.

- The Commission still seems to underestimate the magnitude of the task of coordinating MAST projects,

but the Panel welcomes the moves in the MAST-III Programme to allow more realistic funding of the
project co-ordination costs.

- The Panel very much welcomes the greater attention which is being given to dissemination and

exploitation in MAST-III, but believes that the supporting initiatives in this respect are still not vigorous
enough. The Panel suggests that experts or organisations with a broader perspective would often be
more successful than project partners in disseminating the principal outputs from the MAST projects.

Relevance of MAST Programme objectives

- Most of the initial objectives of the MAST Programme (enhancing European competitiveness, developing

means to exploit ocean resources, establishing the S&T basis for management and protection of the
marine environment) are still valid because :
   - Europe is facing even more severe competition in marine industries from the USA and Asiatic
countries than it was ten years ago

   - concerns about over-exploitation of marine living resources are more acute than five years ago

   - possibilities of exploiting deep-sea resources (oil and gas) are now unfolding

   - awareness of the necessity of a better management of European coasts is growing and is becoming a
critical issue in most countries.

**RECOMMENDATIONS**

The principal recommendations of the Panel are:

**•** [The] [ European Union must maintain a Programme with a strong component of marine science and ]
technology if Europe wishes to exploit the oceans' resources while protecting its marine environment.
The Programme should include all themes relevant to the marine environment, such as coastal
management; fisheries and aquaculture; offshore energy; exploitation of marine resources; maritime
transport; etc. These themes have a considerable overlap in their requirements for science and
technology, including:- understanding and synthesising natural processes and systems; observation and
monitoring systems; forecasting systems; exploration and exploitation technology; socio-economic
modelling; and predicting climate change.

Extra efforts should be made to involve industry (and SMEs in particular) in the MAST Programme. The
procedures for proposing, evaluating and awarding contracts for such industrial projects need reexamining with the purpose of achieving considerable simplification and acceleration, and to make the
funding arrangements more attractive.

**77**

- Border lines between MAST and EUROMAR should be better defined, and a closer collaboration

between the two Programmes should be achieved. The panel suggests that projects approved by MAST
(and those rejected because they are too near-market) should be required to submit a brief project
description to the EUROMAR Secretariat for possible participation in the EUROMAR Programme.

- The Panel recommends also that borderlines with other European agencies (such as the European Science

Foundation and the European Space Agency) need better definition so as to encourage cooperation and
synergy, and that closer collaboration should be established between MAST and other international

programmes.

- Further specific initiatives should be taken to promote dissemination of MAST results, such as (1)

Production of a regular MAST Newsletter, (2) Organisation of workshops on specific themes between
scientists and engineers and technicians from industry, (3) Awards of contracts to technical institutes or
engineering companies to perform a critical review of the relevance and potential uses of the scientific
and technical results of MAST projects, and to disseminate them under an appropriate format.

- The MAST Programme should continue to include a mix of large, medium and small projects. However,

in order to combine the advantages of large and small projects, the Panel recommends that the
Commission should make better use of the possibility of selecting a suite of related projects (rather than
one large or very large project), whose partners would be encouraged to exchange ideas and results at
workshops organised by the Commission under the provisions of 'Accompanying Measures' funding.

Some recommendations are also made for further improvements in the procedures and management of the
Programme:

- For larger projects, the scientific coordinator should be assisted by an executive/administrative

coordinator appointed and financially supported by the Programme.

- The Commission should encourage cooperation between European partners to coordinate the use and

building of large marine research facilities (océanographie vessels, submarine systems, computers,
hydrodynamic tunnels etc.).

- The process of selecting projects for funding should be made more transparent, with a full description of

the criteria used, and of their relative weighting.

- The Commission should examine very carefully the exact purpose of the contract negotiation stage, and

having defined its purpose should make this perfectly clear to the project partners. The contract
negotiation stage should also be made as short as possible.

- Procedures for 'post-project review' of MAST contracts should be developed, published and

implemented.

 - The Commission should examine the feasibility of introducing methods to encourage "new blood"

participation in the MAST Programme.

Another recommendation refers to the need to achieve much closer cooperation between MAST and the

Maritime Task Force.

**78**

**II.** **COMMISSION'S RESPONSE**

The Commission welcomes the independent evaluation of the Marine Science and Technology programme
and will take the recommendations into account when implementing in 1997-98 the second tranche of RTD
projects of the on-going programme.

The Commission takes careful note of the argument that a strong component of research on marine science
and technology must be maintained "if Europe wishes to exploit the ocean's resources while protecting its
marine environment",. Although the FP5 proposal may not include a dedicated programme on the European
seas, the various themes highlighted in the evaluation report are likely to be effectively addressed, especially
through 2 of the priority topics put forward in the Commission document "Inventing tomorrow" which sets
out preliminary guidelines for FP5 : "Unlocking the resources of the living world and the ecosystem" and
"Promoting competitive and sustainable growth".

Attracting more participants from industry has been a long-standing concern for Commission services. The
second general call for MAST-III proposals (deadline of 15 October 1996) reveals a clear shift towards topics
in technology, and consequently an increase in the number of industrial participants is expected. More
efforts on this issue will be made in the future, through focusing of research priorities, more systematic
dissemination of results of projects, the reinforcement of ad-hoc schemes in favour of SMEs, and continued
participation in the Task Force "Maritime Systems of the Future". It must be acknowledged nevertheless that
the very concept of MAST since its creation in 1989, with an important component on studies of processes
taking place in the sea, precludes strong participation from industry throughout the whole range of the
programme. On the other hand, the Commission sees great scope to increase the socio-economic relevance
and visibility of the programme, especially in sensitive areas such as coastal zone management, or the
monitoring and operational forecasting of coastal seas.

A special feature of the MAST programme is the existence of regional-scale projects, in the range of 8-12
MECU of EU contribution. Through these activities, the Commission has experience both in monitoring
large single projects, centrally coordinated by a prime contractor, and in clustering normal size contracts
around common objectives. The Commission endorses the panel's recommendation to resort more to the
latter scheme.

Linkage with international programmes and organizations is a vital necessity. MAST has already contributed
to the objectives of i.a. IGBP core projects and intends to take this further. The MAST office is also active
in EUROMAR and has begun a fruitful collaboration with EMaPS, the secretariat of the European Marine
and Polar Boards newly set up under the auspices of ESF.

By implementing a number of supporting initiatives, the Commission has begun to link up European marine
scientists around common concepts and practices in data management and in modelling. The need to address
on a significant scale, by other appropriate initiatives, the coordinated use of large research facilities, is
acknowledged.

79

**III.** **EVALUATION PANEL**

**Prof.** **Henk POSTMA** (Chairman)
_Background:_ chemistry. Former positions: professor in physical and chemical oceanology at the University
of Groningen (1963-86); director (1965-88) of the Netherlands Institute for Sea Research (NIOZ); chairman
for 10 years of the Netherlands Committee for Sea Research; chairman 1973-75 of the Scientific Committee
on Océanographie Research (SCOR). Member and former chairman of the Dutch SCOPE-IGBP
Commission; member of the Dutch Climate Commission. Officer in the Order of Orange-Nassau.

**Mr. Michael W. OWEN** (rapporteur)
_Background:_ engineering science. Chartered engineer. Chairman 1991-95 of the Programme Management
Committee, coastal engineering research programme, Engineering and Physical Sciences Research Council.
Member (1992- ) of the Coastal Engineering Advisory Committee, Institution of Civil Engineers.
Chairman of the steering group of the MAFF research project on joint probabilities of waves, swell, surges
and tides.

**Dr.** **Josefina** **CASTELLVI**
_Background:_ biological sciences. Research professor at "Consejo Superior de Investigaciones Cientificas"
(CSIC). Main former positions: Vice Secretary General for national coordination of RTD programmes in the
"Comision Interministerial de Ciencia y Tecnologia" (1995); director of the "Instituto de Ciencias del Mar",
CSIC (1994); Spanish representative in the Antarctic Treaty (1988); manager of the National Programme of
research in Antarctica (1989-1994).

**Dr. Kari KVESETH**

_Background:_ environmental chemistry. Since 1993: director of the Division of Science and Technology,
Research Council of Norway. Chairperson of the Council, Norwegian Space Centre. Member of the Board,
Norwegian Academy for Technical Sciences. Member of the Armed Forces Council for research policy.
Main previous honorary positions: chairperson of the Board, Institute of marine research; member of the
National Committee for environmental research.

**Prof.** **Pierre PAPON**

_Background:_ physics, applied mathematics. Professor of physics at Ecole Supérieure de Physique et Chimie
Industrielles (ESPCI), Paris. Chairman of the board of Observatoire des Sciences et Techniques (OST) since
1990. Member of the Executive Council (since 1992) and of the Bureau (since 1995) of the European
Science Foundation. 1989-95: president-director general of IFREMER. 1982-86: director general of CNRS.
Former board member of Agence Française pour la Maîtrise de l'Energie, Commissariat à l'Energie
Atomique, Compagnie Générale d'Electricité.

**Mr. Johannes POST**
_Background:_ oceanography, and also: geophysics, meteorology, theoretical physics, applied mathematics.
Experience in off-shore and deep-sea surveying, computer simulation models and computer software for
marine applications. Founding member and managing director of Hydromod Service GmbH, Hannover,
since 1989. Founding member of OSAT GmbH, Heist (1989). Board member of the GEOMAR Enterprise
Association.

**80**

##### **1996 FTVE-YEAR ASSESSMENT** **BIOTECHNOLOGY**

**81**

**I.** **EXECUTIVE SUMMARY**

**Introduction**

The period of this evaluation is the five years from mid 1991 to mid 1996. The framework II BRIDGE
programme covered the period from 1990 to 1993 with an expenditure of lOOMECU. Biotech-I in
framework III extended from 1992 to 1994 and had a budget of 186MECU. The framework IV Biotech-II
programme extends from 1994 to 1998 with a budget of 588MECU. Thus the period under review covers
the end of the BRIDGE Programme, all of Biotech-I and the first part of Biotech-II. The BRIDGE
Programme has been evaluated separately and a comprehensive report is available. The BRIDGE report is
used as the first frame of reference and changes that occurred in Biotech-I are commented on. It is too
early to reach definitive conclusions about Biotech-II. Thus this evaluation has concentrated mainly on the
activities of Biotech-I but with some comments about changes that are noted in Biotech-II. It incorporates
the final evaluation under the Third Framework Programme.

**The main findings of the evaluation**

1. The initial objectives of the programme remain valid in spite of rapid changes that have taken place in
the science and technology. Close attention to the relevance of the programmes needs to continue since
Molecular Genetics and its associated disciplines now provide basic tools in almost all the bio-industries.
It is important to strike a balance in the objectives between the long term commitment required to realise
the benefits and the need for flexibility in response to the advance of the basic science and technology or
the demands of special situations.

2. Whilst remaining true to its initial objectives, the programme has demonstrated the flexibility to change
and has therefore evolved over time. The Panel approves of this ability to change but new targets, or
changes of emphasis within existing targets, need to be publicised earlier and more widely among
scientists in the community so that they can be aware of how their own field of expertise matches the
evolving priorities of the programme areas.

3. One change that the Panel noted was that Biotech-I was oriented more towards basic research compared
to the earlier and following programmes. Whether due to this or some other cause there was a lower
participation by industry in that programme.

4. The selection criteria are followed in the main and continue to contribute to the building of a European
scientific community although it appears that the cohesion criterion is not so obviously used. There is now
little or no evidence of resistance to trans-national collaboration among scientists. However it is still the
case that many projects are not truly interdependent being more a collection of largely independent subprojects. While such projects do frequently contribute to skill and technology transfer between partners,
the opportunity to deepen further the relationships between European laboratories may be being missed.

5. With the parameters within which it operates, the programme is highly cost effective, having an impact
far greater than its expenditure would predict when compared with the total European effort in this field.
To achieve this it relies on the infrastructure and basic science support of the National programmes.

6. The performance of the Scientific Officers continues to receive favourable comments but there are a
number of areas in which the management process could be improved. These are discussed and
recommendations made. Of particular importance, however, is the delay in starting projects after the
initial funding decisions are made since this can affect the competitiveness of industry and may inhibit
industrial participation.

7. As the programmes have enlarged, the role of the Programme Committee has changed. There is less
clarity in the relationship of the Committee to the evaluating panels. Although the Committee approves
policy, its role in implementation has become more passive. While there is no case for it to become

**83**

involved in evaluation there is a need to ensure that the committee can monitor the link between policy and
implementation.

8. The short term performance of the programme assessed by its publication record seems to be good,
even though all publications may not yet have appeared. However, real time analysis of the progress of
each project against its own objectives would be a valuable addition to performance evaluation and new
project selection criteria. Effects on industrial competitiveness and results in the market will not be seen
until some time after the end of a programme. It will be necessary to evaluate these features on a much
longer time frame. However, it is already clear that the number of patents filed by projects in Biotech-I is
very low.

9. The Panel has found that some companies have difficulties with the rules of ownership of the results of
projects that may have an impact on exploitation. Since this may not be a universal problem an
investigation is needed to assess its potential impact and guide the development of procedures that will
eliminate it.

10. The Panel has found evidence of a need for a broader consultation with industry in the formulation of
programme objectives. In particular, the high level advice provided by IRDAC, the SAGB and the major
trade associations may not adequately represent the experience of small companies nor of the scientific
middle management of large companies.

11. The Panel has noted that there are differences in the needs and methods of operation of small and large
companies. The structure of the present programmes seems to support the needs of large companies well
since they can take a longer term view. Small firms need mechanisms that support their nearer-market
approach. Separate instruments that address their knowledge requirements more directly should be
investigated.

12. As a result of its own experience, the Panel is very concerned about the inefficiency of the procedure
for external evaluation. The period preceding the appointment of the Panel was subject to very extensive
administrative delays that left the Panel a mere three months in which to perform its task. This means that
the analysis could not be based on independent data collected at the instigation of the Panel and at its
worst, could call into question value of the whole exercise.

13. The Panel has recognised a number of major achievements of the programme such as the Lactic Acid
Bacteria project, the Lipase project and the genome sequencing and structure projects, particularly the
Yeast Genome sequencing project. It is noticeable that some projects that are leading to industrial
application require support in more than one programme to achieve success. The mechanism of the
continuation of support through competitive bids of a project that has already been approved and is a
success, may need to be reviewed. The real time monitoring of projects in progress could be of
importance in this respect.

14. The Panel has noted that there is very little co-ordination between the EU and National programmes in
biotechnology, and even less between the various programmes of the Commission. Some duplication or
overlap between Community and National programmes is unavoidable. It even has some advantages and
it may be essential if Community programmes continue to build on the infrastructure and basic scientific
capabilities resulting from the National programmes. On the other hand, the lack of co-ordination
between the biotechnology components of the various EU programmes is inefficient and wasteful of
resources. The best way to overcome this is to combine the biotechnology aspects of all three current
programmes (FAIR, BIOMED and Biotech-II).

15. The EU programmes are continuing to support the scientific base in that a real scientific community
has emerged and continues to be strengthened. The programmes also add value to the European research
effort through projects that could not be co-ordinated effectively by National programmes alone.

**Principal recommendations**

**84**

The Panel has made a set of recommendations under five headings that are set out in chapter IV of this
document . The ten principal recommendations from this set are given below.

1. Since there is evidence that the distribution of biotechnology topics between three programmes
(Biotech-II, Agriculture and Fisheries and **BIOMED-II)** is resulting in a loss of synergy we recommend
that the biotechnology aspects of the current programmes be combined in future programmes but with
clear linkages to the user sectors concerned.

2. As a means of addressing the special problems of small companies and to stimulate processes that can
lead to the initiation of more start-up companies, we recommend the development of financial platforms,
analogous to or alongside the present industrial platforms, with the aim of channelling private sector
funding into the exploitation of opportunities generated by EU programmes.

3. The Panel is of the opinion that the procedures for consulting with industry are too narrow and at too
high a managerial level. We recommend that the consultation process should be modified to take into
account the views of middle management industrial scientists, the views of users and of the managers from
small companies.

4. The Panel confirms its agreement with the conclusion of previous evaluations that training is one of the
most important features of the programmes. We note that while the training component of Biotech-I was
not improved compared to BRIDGE, that of Biotech-II up to the present, is better. We recommend that
the training activity of Biotech-II be carefully monitored to maintain this improved performance.

5. The Panel considers the process of project evaluation and selection to be fair and supports the strong
adherence to peer review and the ranking of projects by scientific quality. The Panel makes a number of
recommendations to improve the process based on thé comments of scientists that have been directly
involved. Of these, probably the most important is that greater emphasis should be given to the quality of
the feedback notes. This is one of the main contacts between Commission staff and the scientific

community and it can be of immense value in the preparation of future applications.

6. The Panel have repeated the observation of earlier panels that the internal Commission procedures for
approving project selection following scientific peer review evaluation are unacceptably long and severely
reduce the competitive impact of the programme. We recommend that a radical review of procedures
should be undertaken to eliminate delays and to reduce the administrative burden on Commission staff.

7. The Panel has observed that projects that support infrastructure activities, such as databases and culture
collections, suffer from difficulties due to the rigid contractual arrangements of the Commission and the
need for the continual renewal of short term contracts for the support of a long term function. We
recommend that a new mechanism be devised to provide long term contracts for infrastructure services
including built in reviewing procedures to check their performance as a justification for the continuation of
funding.

8. The Panel has received evidence that the Programme Committee has become remote from the process^
of project selection and that this separation may permit inadvertent changes of policy due to the spread of
projects selected. We recommend that the Programme Committee be given a quality assurance role in the
process of project selection so that the link between policy and practice is fully monitored by the
Committee.

9. As a result of its experience in the initiation of this evaluation and because of its knowledge of the
preceding administrative delays, the Panel is seriously concerned that the actual evaluation of programmes
is not given proper priority. We recommend an independent, in depth review of the external evaluation
process compared to the Commission objectives, to include the difficulties experienced by this Panel as a

The full report of the panel can be obtained separately.

85

result of the implementation of the current exercise and the use that could be made of real time monitoring
of projects in programme evaluation.

10. The present practice leaves evaluation panels, and perhaps others, unaware of the use that is made of
the results of their efforts. We recommend that the Commission should make an explicit public response
to the recommendations made by external evaluating panels with reasons for their acceptance or rejection.

86

**II.** **COMMISSION'S RESPONSE**

The Commission wishes to express its appreciation to the panel.for its report on the assessment of the
activities of the Biotechnology Programmes for the last five years (1991-1995). The Commission
recognises the fact that the report was carried out under a pressure of time, however, it regards the
outcome as fair, accurate and as having reached balanced conclusions and important recommendations.
The Commission will comment on the main recommendations:

- As the panel recommends, the biotechnology aspects of current programmes should be combined in
future programmes. The future programme covering Life Sciences and the Ecosystem can focus on the
opportunities clarified in the past, so as to reap the benefits of a concentrated effort combining all
resources and means which may be obtained in a single unified programme.

- In addressing the special problems of SMEs, a set of 'stimulation measures for SMEs' has already been
introduced. There seems to be a general appreciation for these measures and the way in which they are
implemented in the Biotechnology programme. A continuous evaluation and further development of these
measures is needed in order to optimise them.

- Concerning industry, the Commission receives advice from IRDAC (Industrial Research and
Development Advisory Committee), especially for the preparation of the Framework Programmes. In
addition to this institutional and strategic advice provided by IRDAC, the Commission services receive
specific advice from other industrial groups throughout the crucial steps of programme preparation and
implementation.

- As far as training is concerned, the Commission is making every effort to maintain the current success
rates which are the result of the recommendations made by the evaluation of BRIDGE. In addition, we are
devising methods for monitoring the quality and outcome of the increased number of grants.

- The Panel considers the project evaluation and selection to be fair. In addition it gives great emphasis on
the quality of the summary consensus reports which serve as a feedback communication mechanism to the
applicants. The Commission takes this remark very seriously. Currently, the quality of the summary
consensus reports have significantly improved compared to the reports of the previous call for proposals.
The importance of adequate feed-back to proposers is emphasised to the external evaluators participating
in the different peer-review panels.

- Since one of the main goals of the Framework Programme is to enhance the competitiveness of European
Industry, measures are being taken in order to increase the transfer of successful research results into
marketable products. The involvement of industrial experts, as well as experts from the industrial
platforms, in the preparation of the programme and in the selection of projects, constitute means for the
promotion of the industrial use of research results.

- The Panel made the observation that the internal Commission procedures concerning project evaluation
and selection are rather long. The real delays for the Commission's decision after peer reviews are
approximately two and a half months. The Commission is making a great effort to reduce it further, but
without detriment to the quality and fairness of the work.

- The Panel suggested the improvement of the coordination among EU and national programmes in
biotechnology. The Commission has already taken measures to do this, aiming at increasing programme
efficiency, synergy and added value. An example is the pilot study dealing with protein engineering and
structural biology activities in Europe, conducted by the Biotechnology programme.

- The Panel has received evidence that the Programme Committee has become remote from the process of
project selection. According to the Commission, there is no reason to believe that the role of the
committee has changed. Indeed, Article 7 of the Council decision stipulates what the committee's role

87

consists of. This has not significantly changed over time, except for the threshold of financial commitment
under which the committee's opinion is not required (now 0.5 MECU, 0.3 MECU before). The
Committees' opinion is now also sought on training grants and all accompanying measures.

- The Panel is seriously concerned that the actual evaluation of Programmes is not given proper priority.
This statement is based on the fact that the administrative delays are by far longer than the actual time
given to the Panel to finalise their exercise. The Commission acknowledges and regrets the time lost
before the panel could carry out its work. It is to be expected however that further assessment and
monitoring exercises will benefit from the fact that monitoring and assessment have now become a
systematic part of the daily routine of the Commission's tasks in the framework of the implementation of
RTD programmes.

- The Panel suggests that the Commission should make an explicit public response to their
recommendations. The present note serves exactly as that; a response to the recommendations made by the
Panel. The Commission finds these recommendations valuable and timely, since the debate and reflection
on the Fifth Framework Programme has started.

88

**III.** **EVALUATION PANEL**

**Prof.** **Dr.** **Rudolf CASPER**

Prof. Casper participated in the Panel for Monitoring 1995 for the Biotechnology Programme. He was the
Director of the Institute of Biochemistry and Plant Virology of the Federal Biological Research Centre for
Agriculture and Forestry, Braunschweig, Germany until 1995. This institute is responsible for the
evaluation and admission of field experiments with transgenic organisms in Germany.

**Dr Luke Gregory GEORGIOU**
Dr Georgiou is the Executive Director of PREST and has experience in evaluations, science policy and
technological innovation. He is a founder member of Commonwealth Consultative Group on Technology
Management and consultant to the UK Technology Policy.

**Dr Marti LEISOLA**

Dr Leisola is the Technology Director of process technology at the Technology Development Centre,
which coordinates and finances applied, technical and industrial R & D in Finland. "He has experience in
evaluating national and international R & D programmes.

M. **Paskal** **BYE**

Mr. Byé, Director of Research at INRA (Institut National de Recherche Agronomique) participated in the
evaluation of the French "Biotechnologies" programme which covered a 10 year period (1982-92). This
evaluation was organized by the CNER (Comité National d'Evaluation de la Recherche).

**Prof.** **Giampiero** **RAMPONI**

Prof. Ramponi is a Director of the Institute of Biological Chemistry of the University of Florence. He is a
member of the progress and evaluation for the group on Biotechnology and Bioinstrumentation of the
CNR.

**Prof.** **Luis NAVARRO**

Prof. L. Navarro has been the manager of the National Programme for Agricultural Research in the
Interministerial Commission for Science and Technology (CICYT). He has collaborated with FAO, the
Hunan Horticultural Institute in China and the Agriculture Ministry of Cuba.

**Prof.** **Gûnther KREIL**

As well as his academic activities, Prof. Kreil has served in the EMBL Council, the EMBO fellowship
committee at the Max Planck Institute for Biochemistry (Munich) and for Experimental Medicine
(Gôttingen).

89

##### **1996 FIVE-YEAR ASSESSMENT** **BIOMEDIÇINE AND HEALTH**

**91**

###### **T. FmirnvF. SUMMARY**

We have reviewed the last five years of the European Union Biomedicine and Health Research Programme.
The Review encompasses the three calls of Biomed 1, under the Third Framework Programme [1], and the first
call of Biomed 2 under the Fourth Framework Programme. Our assessment of the Programme and our
conclusions and recommendations are based on data provided by the CEC, various monitoring reports of the
Biomed 1 and Biomed 2 Programmes and the analysis of data generated by a questionnaire which attracted a
response of 44% from Biomed 1 co-ordinators and 25% from Biomed 2 co-ordinators.
The Research Topics of Biomed 1 were: Pharmaceuticals; Occupational and Environmental Health;
Biomedical Technology; Public Health/Health Service Research; AIDS, TB, Infectious Diseases; Cancer
Research; Cardiovascular Research; Brain Research; Chronic Diseases, Ageing and Rare Diseases; Human
Genome Research; Biomedical Ethics. In Biomed 2 the topics were similar with the addition of 'Legal
aspects and horizontal activities' including demonstration projects as a separate topic.
Both programmes attracted a large number of applicants; 1640 in the case of Biomed 1 of which 403 (25%)
proposals were funded and 1709 in the first call of Biomed 2 of which 302 (18%) were funded. In Biomed 1
the largest number of proposals were submitted in the Research Areas of Brain Research, Cancer Research,
Cardiovascular Research, Chronic and Rare Diseases and Ageing, and Human Genome Research. The
proportion of applications to Brain Research, Chronic and Rare Diseases and Ageing and Public
Health/Health Service Research was increased in Biomed 2 relative to Biomed 1. In Biomed 1, concerted
action (CA) was the main implementation mode, the only exception being Human Genome Research in which
the CEC did fund shared cost (SC) proposals. In Biomed 2 the predominant mode of implementation was
SC.

The total sum requested in Biomed 1 was MECU 824 and the amount awarded was MECU 134. In Biomed
2, MECU 317 were requested and MECU 146 were awarded. This reflects an increase from 16% to 46%
awarded relative to that requested in Biomed 1 compared with Biomed 2, respectively. The average EC
contributions per contract in Biomed 1 and Biomed 2 (first call), respectively, were KECU 304 and 350 for
CA and 585 and 614 for SC.

We conclude that the Biomedicine Programme (Biomed 1 and First Call of Biomed 2) has achieved the
objective of contributing significantly to the improvement of medical and health research and development in
Europe by facilitating the establishment of new collaborations and/or consolidating and strengthening existing
collaborations. The Programme has satisfied the criteria of 'European added value', 'European dimension',
and 'subsidiarity' and is characterised by a multidisciplinary as well as multinational approach which makes it
distinct from Biomedical programmes supported by National Research Bodies and Industry.
The output of the projects supported by the Programme, as assessed by the analysis of Biomed 1, would
appear to be qualitatively and quantitatively equal to or better than that of many projects supported by top
National Funding Bodies. The shift from concertation between many partners in Biomed 1 to actual
research projects between smaller groups of complementary partners in Biomed 2 is a highly positive trend
for which the CEC is to be commended.

The Biomed Programme has also established links with other Programmes of the European Union;
integration with PECO and INCO [3] has promoted participation of non-Member States from Central and
Eastern Europe.
Notwithstanding the success of the Programme, we are concerned by the relatively massive oversubscription
and therefore wastage, due to the limited budget, of a large amount of European talent and expertise.

This 5-year assessment incorporates the final evaluation of the Biomedicine and Health Programme under the Third Framework Programme

Pays de l'Europe centrale et orientale (Central and Eastern European Countries)

Specific Programme for International Co-operation (in the 4th Framework Programme)

93

The method of selection of proposals in Biomed 2, based on highly efficient and effective scientific peer
review, is a marked improvement on the methods of selection used in Biomed 1. Nonetheless further
improvement is possible, and we therefore endorse the 1995 Biomed 2 External Monitoring Report which
recommended that (i) unless there are exceptionally compelling circumstances, proposals should be funded in
strict rank order irrespective of whether or not subareas were represented, and (ii) that the funding of
proposals should be guided by scientific advice and follow a stepwise or sliding scale proportional to the rank
order of the final score of the proposal.
We are sensitive to the need for the strategic, national and political factors that may of necessity influence the
selection of proposals in various areas. A balance could be achieved between the funding of 'response mode'
_versus_ 'strategic top-down' driven research if some degree of science-led flexibility (say 10%) was introduced
in determining the size of the financial envelopes that are set around the various Research Topic Areas by the
CEC and the Member States. Nonetheless, in order to maintain the apparent high quality of the outcome of
the Biomed Programme, and to ensure equity at a time when research funds are exceedingly scarce, we
recommend that irrespective of the strategic importance of any given area, the CEC should establish a
minimum score below which no proposal can be funded. In this respect we commend the Commission for
implementing adequate monitoring systems in the form of the Project Review Board and external monitoring
panels.

Five aspects of the Programme deserve special attention. These are: (i) the relatively small proportion of
industrial contractors; (ii) the relatively small number of proposals concerned with research on 'Illicit Drugs';
(iii) the long delay between the receipt of proposals and the transfer of funds (median of 12 to 16 months);
(iv) the few female expert evaluators and members of monitoring panels; (v) the shortage of CEC Staff
involved in the Biomedicine Programme. Analysis of these issues and recommendations of how any
deficiencies might be rectified are discussed in the text and in the Conclusions and Recommendations.
We highly commend the EU for establishing the Biomedicine Programme which has a major impact on
European Biomedical Research, and will have an important positive effect on the prevention, diagnosis,
management and treatment of diseases in Europe. The Programme provides the EU with an incredibly
powerful network and database of unrivalled expertise in Medicine from molecules through to man and
Society, which in addition to fostering highly productive and integrated research between the Members
States, serves as a vehicle for the rapid deployment of European Task forces to deal with strategic, and/or
national issues as they arise. We very much hope that the Biomedicine Programmes of the Third and Fourth
Framework will form the platform for a similar programme in the Fifth Framework, and that such a
Programme is allocated a significantly larger budget in order to reduce the wastage of many excellent
initiatives which reflect the Biomedical power of the Member States.

**Conclusion and Recommendations**

(I) We conclude that the Biomed Programme (Biomed 1 and First Call of Biomed 2) has achieved the
objective of contributing to the improvement of medical and health research and development in
Europe by facilitating the establishment of new collaborations and/or consolidating and strengthening
existing collaborations in Europe. The Programme has succeeded in satisfying the criteria of
'European added value', 'European dimension', and 'subsidiarity' and is characterised by a
multidisciplinary as well as multinational approach.

(ii) As a consequence, the research supported by the EU Biomedicine Programme is quite distinct from
the Biomedical programmes supported by National Research Bodies and Industry.

(iii) In addition to promoting the formation of new and major European Biomedical consortia, the output
of the projects supported by the Programme, as assessed by the analysis of Biomed 1, appear to be of
a quality and quantity equal to or greater than those of many projects supported by National Bodies.
This assertion is based on (i) the positive nature of the new links forged between partners, including
industrial partners, (ii) contributions to novel methods in health care, (iii) the relatively high number

94

of patents that have been filed, granted and licensed (i.e. contributions to wealth creation), and (iv) the
high impact of the publications produced by a large proportion of the projects supported by the
Biomed Programme.

(iv) As a consequence of CEC policy and a substantial increase in budget, there was a major shift from
Concerted Actions and Centralised Facilities in Biomed 1 to Shared-Cost projects as the predominant
implementation mode in Biomed 2. This shift from concertation between many partners to actual
research projects between a smaller group of complementary partners is a highly positive trend for
which the CEC is to be commended.

(v) Notwithstanding this positive trend, we are concerned by the fact that the relatively limited budget of
the Programme (albeit increasing over the years) meant that only a minority of proposals could be
funded in Biomed 1, and even less in Biomed 2. That is, the level of oversubscription was relatively
high with less than 15% of proposals funded (Biomed 2, first call) in Biomedical Technology, Brain
Research and Cancer Research, and a mean of only 18% of proposals funded for the Programme
overall. Furthermore, the level of funding for each selected proposal is variable, and in some cases
might not allow achievement of all the original objectives that had served as a basis for selection of the
projects. The apparent positive outcome of Biomed 1 (see 3 above) suggests that for CAs, at least,
negotiations between the CEC and the contractors led to a formula whereby CEC together with
National or other (e.g. Industrial) contributions ensured the success of the research proposed. The
outcome of Biomed 2, in which SC is the predominant mode of implementation, has yet to be
determined.

(vi) The Biomed Programme has also established links with other Programmes of the European Union;
integration with PECO and INCO has favoured participation of non-Member States from Central and
Eastern Europe.

(vii) Monitoring systems are in place in the form of the Project Review Board which reviews annually the
progress of each project and has the power to implement changes or terminate unsatisfactory projects
on the basis of written submissions and, if necessary, site visits. The CEC has also complied with the
EU requirements for detailed monitoring of the Programme as a whole of which the present and the
1995 External Monitoring Report of the First Call of Biomed 2 are examples. Nonetheless, there is
room for improvement in that monitoring could be more rigorous, systematic and accessible. We
recommend that the PRB should continue to monitor the scientific progress of the Programme, but in
addition should also monitor the impact of the funding of the Proposals and Areas with respect to the
Output of the Programme. The administrative monitoring of the progress of contracts should be
carried out by dedicated CEC personnel using computerised databases. This would facilitate more
direct and rapid evaluation of changes which are made during the Programme, alert the CEC to the
need for additional changes and facilitate greatly the work of the external monitoring panels.

(viii) The method of selection of proposals in Biomed 2 is a marked improvement on the methods used in
Biomed 1. We strongly commend the CEC for implementing the new Biomed 2 selection method
which entails highly efficient and effective peer review and at the same time secures contractual
confidentiality and secrecy. The latter is important for industrial participation and, by providing
proponents with an assurance of confidentiality, may have played a significant role in the continued
high response rate to the call for proposals in Biomed 2.

(ix) We endorse the 1995 Biomed 2 External Monitoring Report which recommended that (a) unless
there are exceptionally compelling circumstances, proposals should be funded in strict rank order
irrespective of whether or not subareas are represented, and (b) that the funding of proposals should
be guided by scientific advice and follow a stepwise or sliding scale proportional to the rank order of
the scores of the proposals. It remains to be seen whether these two principles are implemented for
the second and third calls of Biomed 2.

95

(x) Notwithstanding this recommendation (ix) we are sensitive to the need for the strategic, national and
political factors that may of necessity influence the selection of proposals in various areas depending
upon (a) the special needs of the EU, (b) the occurrence of apparently new diseases (e.g. AIDS), (c)
the recrudescence of known diseases (e.g. tuberculosis), (d) the emergence or prominence of known
diseases due to changes in demography (e.g. Alzheimer's Type Dementia and other age-related
disorders) and/or (e) novel developments which may promote health care or wealth creation. We,
therefore, recommend that the allocation of funds to Research Areas should ensure that the best
science is funded whenever possible and at the same time allow strategic considerations to be taken
into account. A balance could be achieved between the funding of 'strategic top-down' _versus_
'response mode' driven research if some degree of science-led flexibility (say 10%) was introduced in
determining the size of the financial envelopes that are set around the various Research Topic Areas.
The size of the financial envelopes would still be determined by the CEC and the Member States on
the basis of strategic considerations, but could be adjusted by the Programme Committee in response
to the quality of the proposals in the different Research Areas . This would allow a degree of freedom
in shifting funds between Areas to facilitate changes in emphasis or success in European Biomedical
Research, while at the same time ensure that those Biomedical Areas of greatest strategic importance
for the Member States would be represented in any particular call for research proposals.

(xi) However, in order to maintain the apparent high quality of the outcome of the Biomed Programme,
and to ensure equity at a time when research funds are exceedingly scarce, we strongly recommend
that irrespective of the strategic importance of any given area, the CEC should establish a minimum
score below which no proposal can be funded. Using the present Biomed 2 scoring system (0 to 4,
where 4 is excellent) and the analysis of the first call of Biomed 2 (see 1995 External Monitoring
Report, Figures 12 and 13), the minimum average score for the funding of proposals should be set at
3, which is consonant with the CEC's own criteria of the borderline between 'excellent' and 'good'.

(xii) The proportion of industrial participation, and particularly of SMEs (2.3% of contractors) is perhaps
not as high as might have been wished or expected. The precise reasons for this are not clear, but one
contributing factor may have been the duration of the time taken for the completion of the selection and
funding procedure which ranged from a median of 12 to 16 months. Furthermore, SMEs may be
concerned by the fact that although "confidential", proposals are nonetheless exposed to extensive
peer review and are not assured of success in the competition. Nonetheless, all of the 16 companies
who were partners of respondents to our questionnaire scored the project as important for the business
strategy of their companies, and 9 out of 16 (56%) scored the project to be 'very important' (top
possible score) for their business strategy. The CEC should take this as a commendation from Industry
and as an encouragement to explore ways of fostering greater industrial participation.
We also note that the problem of relatively few SME participants may be partly offset by the fact that
in parallel with the main Biomed Programme, the Commission, in 1995, initiated a scheme for
supporting Exploratory Awards for SMEs as well as Co-operative Research Projects for consortia of
4 SMEs. The impact of this SME Initiative for the Biomed Programme will need to be evaluated at a
later date. However, we recommend that the CEC ensures that the advantages for SMEs in
participating in the mainstream Biomed Programme should be widely advertised (i.e. given greater
visibility).

(xiii) The Programme's work on illicit drugs is not as visible as the EU might have wished. This may be
due to the fact that 'Illicit Drugs" is not a research topic in its own right, but rather a component of
several different Research Areas. However, there are significant positive scientific advantages derived
from the integration of 'illicit drugs' in 'Pharmaceuticals', 'Brain research' and 'Research on AIDS',
for example. Given the high level of oversubscription of the Programme and the relatively small
budget for Biomedicine, establishment of 'Illicit Drugs' as a separate Research Topic might be
deleterious to the Programme as a whole unless it was matched by an appropriate increase in budget.

96

9.

We recommend that the present system is retained unless special, additional sums are assigned for
specific work on illicit drugs.

(xiv) We are concerned by the relatively long delay between the receipt of proposals and the transfer of
funds for research, especially since this may be one factor which might deter Industry from
participating in the Programme. However, we are sensitive to the fact that this delay is due mainly to
the time taken by the negotiation phase which given the small number of CEC Staff and the
complexity of the negotiations is of necessity quite long. This delay is likely to be reduced
significantly if the number of CEC staff was increased from 18 to 28, the number originally allocated
to the Programme (see below).

(xv) There is a deficiency in affirmative action in the Programme in that few female scientists are
appointed as experts for the selection process and in the external monitoring teams. We recommend
that the CEC takes positive steps to rectify this situation and makes every effort to become a successful
affirmative action employer.

(xvi) We have all participated in evaluation and monitoring processes and thereby witnessed at first hand
the excessive workload of the CEC Staff involved in the Biomedicine Programme. The CEC Staff, as
well as having to deal with Programme Calls, evaluations and external monitoring Committees, also
have to manage effectively and efficiently 700, and by the end of 1997, about 1200 contracts. The
contracts involve complex scientific and financial issues in several to many dispirit Member States.
We believe that the excessive workload, which is due in part to the relatively small number of Staff, is
not consistent with either good and humane work practice or efficient and effective Programme
Management. We therefore strongly recommend that because of its central importance to European
Biomedical Science, Health Care and Wealth Creation and because of its obvious popularity, the
Biomedicine Programme should be awarded the 28 staff which it was originally allocated.

(xvii) Notwithstanding some of the concerns mentioned above, we highly commend the EU for
establishing the Biomedicine Programme which has already had a major impact on European
Biomedical Research, and will make significant contributions to the prevention, diagnosis,
management and treatment of diseases in Europe. Furthermore, although the proportion of Industrial
Partners may not be as large as originally hoped for, the obvious importance of the Programme for
industrial contractors suggests that with further targeting of the private sector, the Biomedicine
Programme could also play a significant role in wealth creation.
Our independent analysis has shown that the Biomedicine Programme has a high European
Dimension and Added Value and, that the output of the Programme in terms of new European
initiatives, health care developments, potential wealth creation and quality of publications is generally
excellent. The Programme provides the EU with an incredibly powerful network and database of
unrivalled expertise in Medicine from molecules through to man and Society, which in addition to
fostering highly productive and integrated research between the Members States, serves as a vehicle
for the rapid deployment of European Task forces to deal with strategic, and/or national issues or
crises as they arise. The Programme can already justly and proudly claim major successes such as
the discovery of a new variant of Creutzfeldt-Jakob Disease, the improvement of the survival rate to_
70% of a previously fatal disorder (severe combined immunodeficiency) and many others which are
exemplified in our Report. The high standard, efficiency and efficacy of the Programme depends in
large measure on the work of the CEC staff who deserve special commendation. We have identified
several areas in which technical improvements can still be made, but the Programme provides the EU
with a model for other research initiatives pivotal for the growth and development of Europe.
The Biomedicine Programme offers an important mechanism by means of which individual
biomedical talents in Member States can be harnessed for the greater good of the European Union.
We very much hope that the Biomedicine Programmes of the Third and Fourth Framework
Programmes will form the platform for a similar programme in the Fifth Framework, and that such a

97

Programme is allocated a significantly larger budget in order to reduce the wastage of many excellent
initiatives which reflect the Biomedical power of the Member States.

98

IT. COMMISSION'S RESPONSE

The Commission notes the Panel's comment that both 3rd and 4th Framework Programme activities under
the BIOMED Programme have achieved the objectives to contribute to the improvement of medical and
health research in Europe and satisfied the criterias of European added value, European dimension and
subsidiarity. The Commission notes the clear distinction the evaluators observe between the BIOMED funded
and nationally funded research which is in strict compliance with the principle of subsidiarity. Regarding the
output of the projects supported by the Programme, the Commission notes the Panel's observation that it
appears to be of a quality and quantity equal to or greater than those of many projects supported by National
Bodies.

The trend towards more high quality Shared Cost Actions being proposed and selected for funding is
recognized by the Commission. This is taken as an indication of the increasing demand of actual research
funding on the European level. Although the overall success rate of the programme, in terms of number of
projects, is similar to NIH (USA) and other funding bodies, the Commission shares the concern of the Panel
regarding the oversubscription.

Regarding the recommendation on the allocation of funds to the various research areas, the Commission
notes that indicative budgets for the Programme established by the Council can be and have been amended,
depending upon research priorities, as a specific response to a call and following a positive opinion of the
Programme Committee. The Commission also notes that a high priority is set by the Panel for research on
illicit drugs and that this area is seen as an integral part of the biomedical research activities. This is indeed in
compliance with the recent Council positions on the matter.

The Commission is pleased to receive the comment on the improvements regarding the selection procedures,
which are now described as efficient, effective and confidential, and it will continue to improve evaluation
procedures. The recomendation that a minimum score should be established below which no proposal can be
funded has already been implemented in the second call. The analysis of the second call shows marked
increase in the industrial participation however, health care funding agencies (undertakings, hospitals) are a
major participating force in the BIOMED programme.

In terms of positive action towards having more female scientists appointed to the selection procedures, it
must be noted that BIOMED staff is 50% male and 50% female. The Commission has only limited choice in
the academic scientific evaluators. The ratio of male to female in the names proposed by Member States
is 9:1.

The Project Review Board (PRB) is the main instrument relating to the ongoing monitoring of the scientific
progress of the Programme, and the Panel's recommendation to enhance the role of this body is noted.

The small number of scientific staff involved in the Programme administration at present, recognized by the
Panel, is a very serious problem and efforts are being made to find well-qualified staff in the future.

99

**III.** **EVALUATION PANEL**

**Prof.** S. **AYMÉ**

Director of Research at INSERM. A national expert in Human Genetics, Epidemiology and Public Health
issues for genetic diseases (INSERM and CNRS) and on on rare diseases (Ministry of Health). At the
European level, an expert for the Human Genome Project. President of the European Society of Human
Genetics. In US, Advisory board member of the Genome Database. Member of several editorial boards.
Member of the EUROCAT association.

**Prof.** U. **FAUST**

Former Head of the Instutute of Biomedical Technology in Stuttgart and long term member of the
BIOMED Project Rewiev Board. Wide specialities on the biomedical engineering research, specially on
medical imaging, image processing and sensor technology. Several advisory tasks for industrial R&D
projects.

**Prof.** **G. FINK** (Chairman and Rapporteur)
Director of the Medical Research Council's Brain Metabolism Unit, elected Honorary Professor in the
University of Edinburgh in 1984. Professor Fink is responsible for setting the scientific strategy of the
Unit, deployment of the Unit's Staff and management of the Unit's core budget. His personal expertise is
in neuroendocrinology, neuropharmacology and neuroendocrine molecular biology. His opinion is widely
sought as a referee in the area of Neuroscience on both sides of the Atlantic. He served on the Wellcome
Trust Panel for Mental Health and Neuroscience between 1984-1989. His refereeing has involved the
evaluation of several major institutions. He is on the Editorial Board of several major journals. President
of the European Neuroendocrine Association (1991-1995). Member of Council of the European
Neuroscience Association.

**Prof.** **A. GRAVANIS**

Regular reviewer of research proposals funded from the Greek General Secretariat of Research and
Technology and the Board of Biomedical Research, Greek Ministry of Health. Recognized expert on
molecular pharmacology of hormone-dependent tumors. Autocrine/paracrine interactions of neuropeptides
and steroid hormones in the tumors of the Repro-ductive System. Molecular mechanisms of Oncogene
regulation in cell proliferation/differentiation. Pharmacological evaluation of new hormone antagonists in
the hormonal therapy of cancer.

**Prof.** **J.M. MARTIN-MORENO**
He is a Visiting Fellow of the Harvard School of Public Health, an Associate Researcher of the Milanbased European Institute of Oncology and a Professor in the department of Epidemiology and Biostatistics
at the National School of Public Heath in Madrid, Spain, where he is now the Director.
His experience ranges from implementation and evaluation of research projects at a national and
international level to previous chairmanship of the Discipline Oriented Group on Epidemiology and
Clinical Trials within the BIOMED I programme, and current membership of the Project Review Board of
the European Commission's BIOMED programme. He is interested in epidemiological methods, metaanalysis and, particularly, in the nutritional epidemiology of cancer, a subject on which he has published
numerous papers at home and abroad.

100

**Prof.** **L. NOTARANGELO**

Head, Laboratory Unit of Molecular Immunology, Department of Pediatrics, University of Brescia. Senior
Clinical Investigator, Hematology-Immunology and Bone Marrow Transplantation Unit, Dept of
Pediatrics, Univeristy of Brescia. Permanent Member, WHO Advisory Committee on
Primary Immunodeficiencies.Member of the Editorial Board of several Italian and International Pediatric
Journals. Member of the Evaluation Committee, Italian National Health Institute Research Grants (1992).
Member of the External Monitoring Report Committee, Biomed 2 Programme (1995).
His recent contributions include: characterization of the molecular basis of X-linked
immunodeficiency with hyper-IgM, identification of the gene responsible for autosomal recessive Severe
Combined Immune Deficiency,characterization of the molecular basis of X-linked thrombocytopenia, first
European clinical gene therapy trial for adenosine deaminase deficiency. Scientific Secretary, European
Society for Immune Deficiencies.

101

**10.**

##### **1996 FIVE-YEAR ASSESSMENT** **AGRICULTURE AND FISHERIES,** **INCLUDING AGRO-INDUSTRY**

103

**10.**

**ï.** **EXECUTIVE SUMMARY**

The panel was asked to assess research programmes in agriculture, fisheries, forestry and agro-industry
for the five years 1991-96. This period covered all of the Third, and parts of the Second and Fourth
Framework Programmes. These included over 850 projects, with approximatel 675 MECU of EU
funding.

The changing balance of the programme shows a well reasoned evolution. There has been an increasing
emphasis on forestry and fisheries, as well as on food, nutrition and environment. Most of this change has
been policy driven. The panel felt, however, that the research programme should lead policy, as well as
follow it. This means that there should be a more explicit place in the programme for the pursuit of longer
term goals.

The panel undertook a direct assessment of a randomly selected sample of 15% of the projects in AIR.
Projects were scored for progress in relation to scientific goals, potential impact, effectiveness of
dissemination, and linkages created. For all four criteria, at least 82% of projects were judged to have
performed "in line with expectation" or better. Based on the detailed results of this assessment, and the
evidence from existing reports on other programmes, the panel concluded that the programme as a whole
is very successfully meeting its objectives.

The replacement of the separate sectoral programmes of FP2 with the integrated AIR programme of FP3,
and FAIR in FP4 has worked well. Co-ordination between the three DGs concerned appears good.
Linkages to eight parallel programmes are important, but there is little information on how effective they

are.

Despite some excellent examples (e.g. AIR project catalogue) the overall information system on EU
programmes is not well adapted to facilitate analysis and management.

The information base from which to analyse and co-ordinate linkages between the EU programmes and
national programmes is even less satisfactory.

Broadly speaking, the programmes were very well managed. This judgement was supported by the
opinions of 378 project co-ordinators who responded to a survey. The panel has made a number of
suggestions (see Recommendations) for improvement in the management procedures.

One area requiring additional attention is dissemination. Communication of results to the scientific
community appears to be excellent, but communication to intermediate and end-users could be improved.
The FLAIR FLOW model was judged to be a good example.

**Recommendations** :

1. The research agenda should provide for more work on longer term issues, beyond the immediate policy
objectives, and on systems and themes which may not be economic today.
2. Research across the spectrum of activities encompassed in the term "sustainability" should be more
strongly encouraged. This should be done by delineation of areas where increased work is seen to be
necessary, as well as by including a sustainability criterion in the guidelines on projects.
3. There should be increased emphasis on research which responds to the growing consumer demand for
information on and confidence in the integrity of the food chain.
4. The agenda should encourage a proportion of projects which reach out to the leading edge of scientific
development.
5. Linkages to the environmental program need to be increased.
6. Linkages to national programmes need to be improved, initially by developing an effective common

database.

105

10.

7. The present procedure for project submission could be improved by providing for submission of outline
proposals in a pre-screening phase.
8. On completion, each project should be subject to an independent review.
9. More effective dissemination of results is needed. This applies particularly beyond the level of
scientific publication, which seems adequate. The FLAIR FLOW example could be a model in some
areas. Better promotion of existing mechanisms (Mobility, Dissemination) in connection with the
research programme might also be useful.
10. Greater involvement of industrial partners, particularly SMEs, should be encouraged.
11. Concerted Action type activities should be increased.
12. The management of the programme by the Commission Services needs to be strengthened, in some
areas by the deployment of additional staff, and particularly by the investment of more resources in
an efficient Management Information System.

106

**II.** **COMMISSION'S** **RESPONSE**

The Commission welcomes the external evaluation report concerning Community research programmes in
the field of agriculture, fisheries and agro-industry for the five-year period 1991-1996. The report
provides a good insight of the evolution of successive programmes and the Commission is pleased with the
overall positive opinion. The recommendations made by the panel will be considered when implementing
the remaining part of the Fourth Framework Programme and when preparing the Fifth Framework
Programme.

The objectives of Community research programmes in the fields of agriculture, fisheries and agro-industry
have been manyfold but the most crucial aims include: to promote and harmonize trans-European research
and build a strong scientific basis in Europe; to strengthen the competitiveness of European industries in
these sectors, and to provide support to Community policies. The latter objective does not necessarily
imply immediate needs, for which other sources of funding often exist, but mostly involve long-term
policy objectives (e.g. rural development, sustainable exploitation of fisheries resources). Similarly, many
basic research areas such as plant and animal health, generic science for nutritious foods or the biology of
new aquaculture species have been included in previous and current EU programmes. Furthermore, many
cross-links have been established with other specific EU programmes involving some more basic research
aspects related to the Biotechnology, Biomedicine, Energy, Materials and Industrial Technologies,
Environment or Training and Mobility programmes. In all areas of the previous workprogrammes,
scientific and technological innovation stands as the most important criterion for selection and it is strongly
underlined in calls for tender.

Sustainability issues have received increasing attention during the five-year period and are currently
addressed in many areas of the FAIR workprogramme. The sustainability criteria suggested in the report
will be investigated. Linkages to the environmental programme currently exist through interservice
consultations and meetings for specific areas of the workprogramme. In the context of the Fifth
Framework Programme, research in the fields of life sciences and environmental sciences are likely to be
more closely related.

Several attempts have been made to overcome the difficulties and limitations associated with linkages to
national programmes. A directory of fisheries and aquaculture research organizations including their
research activities has been successfully produced and an inventory of agricultural research projects has
been supported in an initiative named AGREP which is an information system for ongoing, publicly
funded research projects currently implemented in 11 EU Member States. Further to the Commission's
communication 'Achieving coordination through cooperation', a CREST ad hoc group has been
established in 1996 for facilitating the exchange of information on national RTD activities. During the
first year of activity, the group has compiled a large amount of relevant national information and has
produced a valuable report. Concertation activities have continuously been strongly encouraged and
priority has always been given to funding of concerted action proposals at the selection stage. The concept
of thematic network has been promoted in the FAIR programme to reinforce concertation activities.
Efforts will be pursued to stimulate high quality concerted actions, particularly in areas of strategic
importance for Europe.

The level of industrial participation in projects strongly depends on the research areas involved. Agroindustrial projects involve almost always at least one industrial partner and show an increasing industrial
participation during the five-year period with a current involvement higher than 25 %. In primary
production areas, industries are usually not technology oriented and therefore have not been very active in
European research programmes. The technology stimulation measures for SMEs which have been put in
place in FAIR have attracted a growing number of proposals and have proved to be an efficient tool for
raising SME participation particularly in the food technology domain.

107

**10.**

Dissemination of results is a vital component of the Community research programmes. While
dissemination to the scientific community has proved to be very effective through scientific journals and
conferences, dissemination to end-users requires more careful attention. Specific networks have been
established in the food and non-food sectors, whereas dissemination in other domains of the programme
relies more on national initiatives and dissemination plans implemented in each project. Dissemination of
results to SMEs will be ensured through the recently established FAIR network of focal points. Consumer
trends in general and the growing consumer demand for information on and confidence in the integrity of
the food chain in particular have been increasingly addressed in successive programmes and are currently
specifically identified in the context of accompanying measures on ethical, legal and social aspects. Indeed
the role which food plays in human health has been given increasing importance starting from a small base
in FLAIR, continuing in AIR and representing now a significant area in FAIR.

Programme management improvements are continuously sought. While informal pre-screening has been
very active for guiding applicants to the most appropriate sources of funding, it is felt that a formal prescreening step could create even bigger disappointments among proposers and that targeting calls for
specific areas could be a more effective mechanism to combat oversubscription. Various ways for
strengthening project monitoring and assessment after completion are also being considered including the
assistance of external experts. Finally the need for an efficient information management system is
recognized by the services. New information systems have been recently developed (e.g. AMPERE) but
further improvement are necessary to allow harmonized and up-to-date information management. This
issue could also be resolved by increasing human resources at the Commission level.

In conclusion, most of the recommendations made by the panel prove to be consistent with the first
orientations for the Fifth Framework Programme, proposed by the Commission in 'Inventing tomorrow'.
The Commission strongly believes that this independent evaluation of EU RTD activities will thus be very
valuable for orienting further implementation of the Fourth Framework Programme and for preparing the
Fifth Framework Programme.

108

**m .** **EVALUATION PANEL**

**Prof.** **Stefano** **Cataudella,** **Università** **degli Studi** **di Tor Vergara (IT)**
Professor Cataudella, Associated Professor in the University Tor Vergara (Rome) has a wide experience in
research, especially in aquaculture-environment relationships and in genetic characterization of reared
population of fish. Concerning academic experience he has been involved in different university
departments since 1973 and co-operated with international programmes for development. He also
participated in the evaluation of FAR projects.

**Prof.** **Patrick Cunningham, Trinity College Dublin (IE), Panel Chairman**
Professor Cunningham was Director of the Animal Production and Health Division at the FAO until 1993.
Since then, he returned to his position as Professor of Animal Genetics at Trinity College and is very
active in various international advisory committees. His research work has been centred on quantitative
genetic theory and on the efficiency of livestock improvement programmes.

**Mr Francisco Javier Pereiro** **Mufioz,** **Instituto Espanol de** **Oceanografia** **(ES)**
Mr Pereiro Mufioz has a twenty-year experience in fisheries research, especially in demersal species in
European waters. He is the head of the IEO Fisheries Research Programmes in European waters and has
been member of the Scientific and Technical Committee for Fisheries (STCF) of the EC. He is Spanish
delegate in some committees of the International Council for the Exploration of the Sea (ICES) and in the
ACFM (Advisory Committee for Fisheries Management).

**Dr Daniel Richard-Molard, Institut National de** **la** **Recherche Agronomique (FR)**
Dr Richard-Molard is currently the Head of the Protein and Carbohydrate Technology department as well
as the Fruit and Vegetable Biotechnology department at INRA. His research work is focused on the
transformation technologies of cereals for food and non-food purposes and involves many cooperation
programmes with industry.

**Prof.** **Jorma Sundquist, The Finnish Pulp and Paper Research Institute** **(FI)**
Professor Sundquist was the Research Director of the Paper Science Centre at the Finnish Pulp and
Research Institute. His research interests cover polymer science, wood and pulping chemistry as well as
chemical textile technology. He also participated in the evaluation of FAIR projects.

**Prof.** **S. Athanasios Tsaftaris, Agricultural University of Thessaloniki (GR)**
Professor Tsaftaris's research interests include many different aspects of plants, particularly biochemistry,
physiology, genetics, breeding and biotechnology. Professor Tsaftaris is currently serving in the National
Research Council of Greece and is a member of several international scientific advisory boards.

**Prof.** **Marcel Vanbelle, Université Catholique de Louvain (BE)**
Professor Vanbelle's area of expertise covers animal and human nutrition. He has been a member of the
Belgian Council of Human Nutrition since 1990, has published more than 450 papers and is a member of
several international scientific advisory boards.

**Prof.** **Heinrich** **Wohlmeyer,** **Agricultural University of Vienna (AT)**
Professor Wohlmeyer is an agricultural scientist with international training in economics and commodity
agreement. An industrial experience in agro-industry preceded his current research position at the
University. His research interests involve natural resource management and environmental economics.

109

**Il**

##### **1996 FIVE-YEAR ASSESSMENT** **NON NUCLEAR ENERGY**

**11!**

**11.**

**I.** **EXECUTIVE SUMMARY**

**1.1 Background**

This is a self-contained summary of the report of the Assessment Panel on the Non-Nuclear Energy (NNE)
RTD&D activities of the EC for the period 1991-1996. The summary forms an integral part of the main
report of the Panel. The Assessment was carried out by a panel of independent experts, whose names and
Terms of Reference are annexed to the main report. These RTD&D activities took place under the JOULE
Programme within the Third Framework Programme and under the THERMIE Programme, and under
their successor, the JOULE-THERMIE Programme, within the Fourth Framework Programme.

As no final evaluation of these programmes was carried out under the Third Framework Programme, this
Assessment subsumes the evaluation of the activities under the Third Framework Programme, in
accordance with the Commission's Communication to Council COM(96) 220 (final) of 22 May, 1996.

**1.2 The** NNE **RTD&D activities**

The oil crises of the seventies spurred the EC interest in energy research and demonstration, with the
research element being managed by DG XII and the demonstration element by DG XVII. The energy
research programme became known as the "JOULE" Programme in 1989 under the Second Framework
Programme, and the demonstration element as the "THERMIE" Programme, which was not part of the
Framework Programme. The two elements were amalgamated to form the "JOULE-THERMIE"
Programme in 1994, under the Fourth Framework Programme. The two elements continue to be formally
administered and managed by DG XII and DG XVII, respectively, and to have two separate management
committees.

The objectives of the two programmes and their successor are described in Section 3.1 and in Annexe 2 of
the main report. As a generalisation, project co-ordinators in the JOULE Programme tend to be located in
universities and research organisations, whereas those in the THERMIE Programme are industrial
enterprises or industrial research institutions. While individual project budgets under THERMIE tend to be
larger than those under JOULE, the overall programme budgets are comparable.

The technical activities within the NNE programmes cover a wide domain including Renewables Energies,
the Rational Use of Energy, and Fossil Fuels. In addition to research and demonstration actions, the
domain includes associated measures such as supporting and dissemination actions, and strategic studies.

**1.3 Structure of the main report**

The main report has been structured in accordance with the format suggested by CREST for the
assessment of all specific RTD&D programmes.

- **Section 1** presents a self-contained summary of the main report.
- **Section 2** is an **introduction** to the report and to the methodology used.
- **Section** 3 provides a detailed **critique of the programmes.**

- The Assessment Panel's **conclusions** are presented in **Section 4.**

- The Panel's **recommendations** are presented in **Section 5.**

- Supplementary information is contained in the **Annexes.**

**1.4 Methodology of the Assessment**

This assessment was carried out at the programme level, not at the project level. A **questionnaire** was sent
by the JOULE and THERMIE managements to a random sample of **340 project co-ordinators.** The
questionnaires were returned directly to the rapporteur over a period of months. Analyses of these

113

**11.**

questionnaires are presented in Annexes 3 and 4. **Background information** on the programmes was
provided by the Commission services to this Panel at the beginning of, and during, the Assessment. The
Assessment panel designed a **questionnaire,** which was distributed, with the permission and co-operation
of the Programme Directors, to **programme managers** and staff responsible for the JOULE and
THERMIE elements of the Programme. Responses were returned anonymously to the rapporteur, and are
presented in summary form in Annexe 5. The Assessment Panel **interviewed** the **Directors** of the JOULE
and THERMIE programmes with some of their staff.

The Assessment Panel first met in Brussels on 2nd September, 1996 and on five occasions between then
and 20th December, 1996.

**1.5 Assessment of the** NNE **RTD&D activities**

By far the greater part of the EU's energy supply still comes from non-nuclear sources and will continue
to do so for the foreseeable future. The programme's concern with non-nuclear energy is therefore still
justified. There remain concerns about security of energy supply, environmental impacts and other issues.
The **objectives** of the Community's actions in the domain of NNE RTD&D were to:

- Increase the long-term security of energy supply

- Reduce consumption and imports of energy

- Alleviate environmental problems related to energy production and consumption.

These objectives were to be served through carrying out RTD&D on the rational use of energy and
exploitation of renewable and new indigenous sources of energy. They were reinforced and broadened to
take account of Union policies and of socio-economic and political changes. The initial objectives of the
programmes are still valid against evolving scientific, technical, industrial and socio-economic conditions.
In the Assessment Panel's opinion they will remain largely valid for the Fifth Framework Programme,
taking account of the availability of energy technologies and of their slow market penetration.

Selection criteria, procedures, objectives and scope of the activities as laid down in Council Directives
have been respected by the Commission. However, it has proved difficult to adopt specific and verifiable
Programme objectives against which performance can be evaluated. This is a common problem in
research.

In the Panel's opinion, the NNE activities have been managed in a cost-effective manner. The staff
resources allocated to management are, if anything, somewhat inadequate resulting in overload and delays.
Nonetheless, the management of the activities is generally satisfactory and efficient. Information provided
to proposers has improved in quality in recent years and the project selection process in the JOULE
element is transparent. Although the NNE activities fall within one programme, the JOULE-THERMIE
Programme, there are in fact two management committees and there are legal separations between the two
elements. An external monitoring process was introduced in 1995 and has proved useful to this Panel and
to the Commission services.

The management of the Programme has appropriate modes of action open to it and adequate flexibility in
operation. However, this freedom has not always been utilised to best advantage.

The Assessment Panel found it difficult to judge the effectiveness of the Programme or the achievement of
objectives. This is a problem common to all research activities which have broadly-stated objectives, and
is compounded by the fact that national governments and private organisations also carry out NNE
RTD&D, making the Commission's achievements difficult to isolate. However, there are areas in all three
main sectors of the Programme (Rational Use of Energy, Renewable Energies and Fossil Fuels) where the
NNE activities have given the Union a leadership position which national efforts would not have done as
well. Particular examples of successful intervention are to be found in fuel cell technology, wind turbine
design and combustion in automotive engines. The Community can play a particularly effective role in

114

11.

supporting long term and risky transnational research projects which individual Member States might have
difficulty in supporting.

Replication and commercialisation have not always been as successful as would be desired, due in part to
the low price of fuels and to the apparent security of supply. The response from contractors to the
questionnaires, although disappointing in some respects, did indicate substantial outputs in the forms of
publications, theses, new products, new processes and test methods.

The scientific and technological base of the Union has undoubtedly been strengthened by the NNE
activities through the development of research teams and research facilities, and the promotion of
international co-operation. In addition, the NNE activities are consistent with the Community's energy
policy and have supported other Community policies on the environment, cohesion, the development of
the internal market and sustainable development. There are in fact at least twelve Community initiatives
and instruments related to the NNE activities which serve to exploit and disseminate research results. The
linkages between these programmes and instruments are still being developed. Coordination between the
Community's activities and those of Member States has not always been good, but is improving.

**1.6 Conclusions**

The Assessment Panel's conclusions are summarised below:

The initial objectives of the programmes are still valid against evolving scientific, technical, industrial
and socio-economic conditions. These objectives will remain largely valid for the Fifth Framework
Programme.
Community-funded RTD&D has contributed to almost all the developments in the field of NNE and
has also helped strengthen the Union's scientifc and technical base. Funding has been provided for
long term and risky research, which might not have been funded otherwise.
_De facto,_ the two elements of the Programme operate largely as two separate programmes, as they

originated in two different Directorate General.
The management of this programme is considered by this Assessment Panel to be cost effective. The
transparency of management is generally satisfactory.

 - Both the THERMIE and JOULE elements of the JOULE-THERMIE Programme appear to be understaffed. This has resulted in delays in contract negotiation and insufficient time for real project
management.
Delays in contract negotiation continue to be a problem for contractors and appear to be caused in
part by delays in the contracts division.
Clear quantified objectives have not been set for the Programme, making if difficult to assess its
effectiveness. It might be difficult to set such objectives.
Co-ordination of Community and national programmes is not satisfactory.
Full use does not appear to have been made of all available modes of action.
The project selection criteria and procedures, as well as the objectives and scope, laid down in
Council directives have been respected by the Commission.
Literature provided for intending participants in the Programme has improved, but there is room for
further improvement. There is also scope for harmonising the procedures for project selection in
JOULE and in THERMIE.

It is difficult to fund good projects which contain elements of research as well as demonstration.
Project evaluations do not fully use metrics such as milestones and outputs.
Evaluation of completed projects within the JOULE element of the Programme has hitherto been nonexistent, but is now in train.

 - The establishment of the annual monitoring exercise is of great assistance to the programme
management.

115

11.

**1.7 Recommendations**

The Assessment Panel recommends:

That RTD&D activities should continue in the area of NNE.

The NNE Programme should be more oriented towards user needs, emphasising applications of
technologies, but a portion of the budget should still be directed to basic research. High quality
standards must be maintained at all times.

Further action should be taken to harmonise the operation of the JOULE and THERMIE elements of
this programme, for example by having a unified management structure and by having harmonised
proposal evaluation and selection procedures.

Better integration of THERMIE type A and type B actions should be assured to improve replication

rates.

Monitoring and evaluation procedures should be extended and improved to include mid-term reviews,
clearly defined milestones and annual reporting with project highlights for both the JOULE and
THERMIE elements. The monitoring panel should have continuity and a member of the monitoring
panel should serve on the Assessment Panel.

More use should be made in the JOULE element of the Programme of all of the modes of action and
of the flexibility open to the programme management, such as concerted actions.

Adequate resources should be devoted to programme and project management, _inter alia_ to reduce
delays in contract negotiation.

Better co-ordination with, and between, Member States' programmes should be established.

A procedure should be devised to deal with good proposals which overlap the JOULE and THERMIE
elements of the Programme.

116

**11.**

**II.** **COMMISSION'S** **RESPONSE**

The Commission welcomes this independent evaluation of Research, Technological Development and
Demonstration (RTD&D) activities in the field of Non-Nuclear Energy (NNE), prepared by the high level
group of experts chaired by Dr. Eamonn Kinsella, and is grateful for the effort shown by the panel in
assessing the last five years of the programme.

The report comments on an important number of elements associated with the relevance of the programme
objectives, efficiency and effectiveness of overall management and implementation mechanisms. The
Commission views the remarks as providing helpful guidance and takes good note of the Assessment
Panel's conclusions that initial objectives of the programme should remain largely valid for the future; that
Community-funded RTD&D has contributed to almost all the developments in the field of NNE and has
also helped strengthen the Union's scientific and technical base; that project selection criteria and
procedures, as well as the objectives and scope, laid down in Council decisions have been respected; and
that the overall management is considered to be cost effective.

The Commission welcomes the Panel's recommendation to orient the NNE programme more towards user
needs, emphasizing the application of technologies. In fact, the provision of high-quality energy
technologies and services - which respond to the citizens expectations for improved quality of life - and the
timely demonstration that these technologies and services will limit environmental hazards are essential
elements of the European Energy policy.

The Commission supports the Panel's view that the objectives of the Community's actions in the domain
of NNE are served through carrying out RTD&D actions on rational use of energy, exploitation of
renewable and new indigenous sources of energy and also in the hydrocarbon sector.

The Commission is also aware that additional efforts are needed, at the whole framework level, for
harmonizing management and administrative procedures as well as improving co-ordination with, and
between, the various Member States' RTD&D programmes. The Communication "Inventing Tomorrow"
and the subsequent Working Document on the Structure, Instruments and Objectives of the Fifth
Framework Programme indicate the main lines of actions which are intended to be followed in this

respect.

Finally, the Commission views the _recommendations_ as very constructive, gearing both future activities
and identifying short term adjustments which are due to be timely introduced in the remaining parts of the

current programme.

117

**11.**

**III.** **EVALUATION PANEL**

**Dr. Eamonn KINSELLA**

Mr. E. Kinsella is Director of a consultance company The Circa Group Europe. He is an expert in energy
and in material technologies. He was a Research manager at Moy Insulation Ltd and a Member of the
National Board for Science & Technology (Ireland).

**Mr. Maurice CLAVERIE**

At the CNRS, Mr. Claverie is responsible for the relations with the Public Research Establishments and
coordinator of the PIR (Programmes Interdisciplinaires de Recherche). He is also advisor at the DGRT
(Direction Générale de la Recherche et de la Technologie) for the energy and environmental problems.

**Mr. Dale LAIDLER**

Mr. D. Laidler has joined ICI Corporate Laboratory as Senior Research Scientist. Since 1991, he is the
External Collaborations Manager. The role involves helping individual research groups within ICI to become
involved in academic and inter-company collaborative activities.

**Mr. Hans Christian SORENSEN**

Director at the Energi og Miljoundersogelse company, Mr. Sorensen is specialised in the field of rational use
of energy and use of renewables in buildings and in the area of combustion.

**Prof.** **Anastasios KARABELAS**

Mr. Karabelas is Professor in the Chemical Enginnering Department of the Aristotle University of
Thessaloniki. He is Director of the Laboratory of Chemical Process and Plant Design.

**Prof.** **Dr.** **Ing.** **Dr.** **Ing.** **E.h. Helmut SCHAEFER**
Mr. Schaeffer is Professor at the Technical University of Munchen. He is responsible for the Research Unit
for Utilities. He has experience in applied research in the area of utilisation of energy in industry, in domestic
and transport (structural analysis in energy demand, energy concepts, ...).

118

12.

##### **1996 FIVE-YEAR ASSESSMENT** **TRANSPORT**

119

**12.**

**T.** **EXECUTIVE SUMMARY**

**Recommendations**

1. **It** is very important that transport remains a distinct programme, while being strongly connected to other
areas of the Framework Programme.

2. The subsequent overall transport RTD programmes should include transport related aspects like
technology development, energy conservation, social and environmental considerations, land use etc, with
the various aspects managed by the appropriate DGs within a coherent framework.

3. It is important to keep a mixture of technology research and policy research within one overall transport
RTD programme.

4. Tasks to be included in the subsequent programmes should be selected so that highest priority is given to
problems that most effectively are handled on an European level and especially those which are of
importance for the EU transport policy.

5. It is important that the overall programme has a clear and communicable vision; there should be scope for
selected risk taking and also some long range research that spans successive Framework Programmes.

6. The specific transport programme should be structured so that there is a strategic part, an "umbrella",
where problems common to the different modes, and co-ordination and competition between modes are
handled. Under this "umbrella" relations between transport and other related aspects should be included.

7. There is an urgent need to increase the number of staff to manage the transport RTD activities; initially up
to the agreed staffing levels but a policy-oriented programme of this nature needs a higher ratio of
scientific officers to projects than in more technology-oriented programmes, so agreed levels should be
raised.

8. There is a need to build in more flexibility in the programming of the budgets. A certain amount of money
should be kept aside within the specific programme to make it possible to react to changes in society and
policy, new developments in technology, to prolong projects from earlier programmes, etc.

9. We strongly support the measures DGVH are proposing to take to improve dissemination; we emphasise
the need for dedicated in-house staff with specific responsibility to facilitate dissemination and recommend
that appropriate resources are made available.

10.Concerted actions and task forces have proved to be very effective in enhancing co-operation and coordination, and these mechanisms should be built appropriately into future Framework Programmes.

**Background**

This assessment report has been prepared according to Article 4.2. of the Council Decision of 15th
December 1994 on the specific RTD programme in the field of Transport. The programmes and initiatives
covered by this 5-year-assessment are: EURET, APAS and the start-up phase of the much larger Transport
RTD Programme in FP4.

The objectives of this five year assessment are to consider the relevance, efficiency and effectiveness of the
programmes and initiatives and the contribution of the RTD activities to meeting the Community's transport
and other policy objectives.

121

12.

This assessment was carried out between July and October 1996. The Panel examined project and evaluation
reports and the documentation associated with the Transport RTD Programme. In addition, the evaluators
have held a number of meetings with project leaders, representatives from related EU projects, members of
the Transport Research Committee (TRC) and staff from DGVII, DG XIII and Eurocontrol. The evaluation
panel met on four occasions at the offices of DGVII, and corresponded directly at other times.

The intention of the transport research programme is to improve the efficiency of the individual transport
modes and to find a strategy for a faster integration and interoperability. The research covers both
passenger and freight transport. There are other specific programmes that are related to transport, which are
evaluated separately, but we have included some connections to these other programmes.

**Major achievements**

It is only from the limited budget EURET and APAS activities that major achievements can be expected,
as the Transport RTD work is still in its early stages. Examples of major achievements include the
following:

**Intermodal:** design of an innovative and efficient system of rapid loading and unloading of goods between
different modes of transport (SIMET).
**Rail:** development of a harmonised European Rail Traffic Management System (ERTMS) aimed at
overcoming the fragmentation of rail traffic management systems used by the national railways.
**Waterborne (maritime):** application of advanced technologies and their integration with optimum crew
composition, manning and operational strategies (ATOMOS).
**Air:** improvement and evaluation of air traffic management scenarios (AEGIS).
**Road:** use of cost-benefit and multi-criteria analysis for new road constructions.

In addition, the research activities have been very successful in encouraging the relevant groups in
industry, transport operators, researchers and policy makers to work more closely together, and in
fostering a greater degree of co-operation between organisations in different Member States.

122

**12.**

**II.** **COMMISSION'S** **RESPONSE**

The Commission welcomes this independent evaluation of EU **RTD** in the field of transport. The
recommendations will be taken into account in orienting the further development of transport related
research in the 4th and 5th Framework Programmes and will also have an impact upon the way in which
the results of the research and development are disseminated and implemented.

The Commission acknowledges the need for a mixture of "technology" research and "policy" research. It
welcomes the panel's conclusion that "the contribution of the specific transport **RTD** programmes to
assisting in the implementation of the transport policy of the EU is considerable". This requirement to
support policy is clearly set out in Article 130f of the Union treaty which refers to the need for research to
strengthen the scientific and technological bases of Community Industry and to support other Community
policies. It is also clearly acknowledged in the Communication from the Commission "Inventing
Tomorrow". [1 ]

The panel desires to see a single Transport RTD Programme in FP5. The need for a degree of
concentration of future research efforts, has been recognised by the Commission. The FP5 proposal is
likely to see a significant reduction in the number of specific programmes. This would facilitate the cooperation and co-ordination of research effort on themes such as transport which are currently situated
within four different specific programmes.

The Commission also notes the opinion of the panel with regard to the need for an overall "umbrella" for
transport RTD under which problems common to the different modes, and competition and co-ordination
between modes, are clearly handled. The specific Transport **RTD** Programme has devoted around 20% of
its budget to Strategic Research which is in part fulfilling such a role.

As regards flexibility, the Commission recognises the need for flexible programming and management in
the context of the Transport RTD Programme. Use has already been made of the limited flexibility
provided for by the specific Programme on Transport. Examples include the modification of the funding
basis for a number of tasks and the emphasis on aspects of research related to road pricing introduced in
the Third Call for Proposals. The creation of greater flexibility, through setting aside a certain percentage
of funds within a specific programme to cover unforseen requirements, will however be considered when
preparing the Fifth Framework Programme.

The role of concerted actions and task forces has rightly been recognised by the panel as a means for
enhancing co-operation and co-ordination. The Commission intends to incorporate both of these types of
actions in future Framework Programmes.

Finally, the Commission recognises the importance of dissemination and exploitation in the field of
transport RTD. It insists, within the Transport RTD Programme, that all projects include a work package
dedicated to the dissemination and exploitation of **RTD** results. In addition, a specific task at a
programme level was launched in the 3rd Call for Proposals on 17 December 1996 which aims at a range
of measures to foster the dissemination and exploitation of all research results.

1 COM (96) 332 final - 10.07.1996

123

**12.**

**IIL** **EVALUATION PANEL**

**Professor Peter Jones**

Director, Transport Studies Group,University of Westminster. **UK**

**Professor Bengt Holmberg**
Professor, Dept. of Traffic Planning and Engineering,
University of Lund **SW**

**Mr Ambrosius Baanders**

Deputy Director,
Transport Research Centre A W **NL**

**Mr Jorge Hernando** **Cogollor**
President, Consultrans S.A. **ES**

**Mr Harri Kallberg**
Director, Road Traffic Centre. **SF**

**Dipl-Ing** **Konrad Lukaschek**
Head of Division, Signalling and Traffic Control Systems
SIEMENS AG Ôsterreich. AU

124

13.

##### **1996 FIVE-YEAR ASSESSMENT** **TARGETED SOCIO-ECONOMIC RESEARCH**

125

**13.**

**T.** **EXECUTIVE SUMMARY**

**1.1** **The** **Panel's** **general findings on the Targeted Socio-economic Research Programme**

Before the start of the 4th Framework Programme, socio-economic research at Community level was
characterised by two processes: On the one hand concrete social science research and studies were
performed in connection with the **MONITOR** programme and as an integral part of some specific
programmes. On the other hand, a broad discussion on the potential importance of collaborative European
social science research started in the late 1980s. Major institutional actors contributed substantially to that
discourse.

The 4th Framework Programme followed the ambitious plans that were the outcome of that intensive
interactive process only to a very limited extent. The scope of the TSER programme is not only much
narrower than the different considerations in the beginning of the nineties have envisaged, but it is also
divergent from them in so far as it is highly fragmented in nature. Thus, it appears more as a first pilot
activity in the area of socio-economic research at European level and less as a strategic tool to investigate
into the deep roots which give way to exclusion, poverty, unemployment, urban deterioration etc. in
Europe.

The assessment Panel has concentrated on the TSER programme as it is, not on TSER as it could or
should have been. In addition, it must be emphasised that the Panel's assessment is based on insights and
impressions obtained at a very early stage of the programme's development, when projects have started
only a few months ago. Keeping these points in mind:

a. despite its limited scope, the TSER programme must be considered as a first important attempt to
improve the knowledge on a number of important specific problems and to encourage large scale cooperation in the European social science community thus providing inputs into the political debate and
contributing to the development of a European socio-economic research infrastructure;
b. although the programme has seen many different challenges, mainly on the general leadership and
management level in addition to the typical learning problems of a fairly new activity, a number of high
quality research projects and thematic networks have been selected and started to work a few months

ago.

The TSER programme is providing a new platform for collaborative multi-national, multi-cultural, policyoriented and policy-relevant research, which is a new feature for socio-economic research in Europe. The
programme objectives are more valid than ever. The Panel expects that relevant results will be produced,
in parts of the programme already in the very near future. The Panel was pleased that in some of the
projects some interesting results are available already. The results of the programme will contribute to the
development of new scientific knowledge in the areas of socio-economic research covered by the
programme, and to the improvement of Community policies in the areas of research and technological
development, education and training, and social exclusion and integration.

Since the programme's specific objectives call for extensive networking as a special characteristic of
TSER, a precondition for safeguarding the success will be that the research/policy links are further
developed - at the level of the programme, the thematic clusters and the individual projects. First results
are promising, but appropriate mechanisms of intensive interaction and communication have to be ensured
and strengthened also in the future.

The TSER programme has been facing many organisational difficulties in its development - already four
different directors in its short life time, changes in personnel and responsibilities, and a number of
different organisational structures. Obviously, the Commission was not able to ensure a harmonious start
up phase of that new activity until recently. For the Programme Committee the leadership problems at the
programme level in the first period of the programme development have been less then satisfactory.

127

13.

However, the Panel acknowledges that during the implementation phase of the programme the
Commission staff performed surprisingly well under very complex, turbulent and sometimes confusing
circumstances, and succeeded to organise the start up of the programme - call for proposals, project
evaluation and selection, contract negotiation - efficiently and to a high standard. The Panel is satisfied to
note that for the second Call for Proposals the recommendations of the 1995 External Monitoring Report
have been followed.

The objectives of the TSER programme as defined in the Council Decision are defined sufficiently clear.
In the future, however, target groups and performance indicators should be defined where appropriate.

The present Panel is aware of the criticism by the previous 1995 External Monitoring Report of the TSER
Programme that the management costs of the TSER Programme are too high. After one year of experience
the Panel takes a different perspective on this issue. The specific nature of the programme in connection
with its important networking tasks calls for adequate staffing - both in numbers and in appropriate
qualifications. In the very near future, the situation will become even worse, when the projects resulting
from the second Call for Proposals will be starting.

The TSER programme has appropriate modes of action - research projects, thematic networks, concerted
actions and accompanying measures. The Panel would welcome that thematic networks and concerted
actions are stronger advertised, because these modes of participation in the programme are especially
suited to mobilise and stimulate European co-operation in the socio-economic research community where
this was not developed so far in the past. In addition, the Panel strongly support the attempts to cluster
projects around themes following similar objectives, thus providing a mechanism for re-integration of an
originally fragmented programme.

The quality of the selected projects leads the Panel to believe that they will generate considerable addedvalue. Thus, the Panel expects a considerable impact of the programme - in economic terms through its
future contribution to the improvement of Community RTD policies, and in terms of inputs to the
development of new European approaches in education and training as well as of new social models. Also
its potential contribution to harmonisation of social data has to be emphasised. The TSER Programme has
initiated a process of Europeanisation in the different areas of socio-economic research addressed so far.
This can be seen as an important impact in itself already, because a well developed European social
science infrastructure will be needed in the future.

However, as has been said already, the achievement of the objectives will to a large extent depend on the
effectiveness and efficiency of the research/policy interaction.

The Panel has identified deficits with regard to the communication, networking and co-ordination tasks of
the TSER programme, in the direction of other specific programmes, other Commission services and of
the policy levels in the Community and in the member states. These deficiencies are clearly linked to the
staffing problem of the programme.

The Panel strongly supports a dual approach to the integration of social sciences in the 5th Framework
Programme:

. The Panel firmly recommends a separate specific socio-economic research programme addressing a
well chosen, coherent set of themes that are European by their very nature. A specific programme will
provide the visibility and the integrative momentum that will be necessary to intensify the formation of
a European social science area, to support other Community policies and to underpin the activities of
other Community programmes and Commission services.
. The Commission should continue and strengthen the social sciences parts of specific programmes where
appropriate. It will be necessary to further develop specific methodologies for risk and impact
assessment in the different technology areas, so as to arrive at integrated planning systems that will help
avoiding negative social, environmental and economic impacts from the outset.

128

13.

The credibility of the European Union as a supra-national organisation will depend to a large extent on its
success in solving the most challenging problems in the different policy areas connected with the overall
process of European integration. The Panel is convinced that there is a need for integrated, systemoriented approaches as provided by a well designed future socio-economic programme.

**1.2** **The Panel's major recommendations**

In the following the Panel's major policy relevant recommendations are given in a condensed form. A
much more detailed list is given at the end of the report.

1.2.1 Contents of the TSER programme

**1.2.1.1** Programme design, workprogramme etc

1. The Panel supports the notion of European multi-cultural, multi-disciplinary, policy-oriented and
policy-relevant socio-economic research organised in a separate specific programme for socioeconomic research.

2. The Panel recommends to discuss in the Programme Committee a revision of the work programme
for the 3rd Call for Proposals, restricting the contents to a few strategic themes and ensuring
synergies between the different areas of the programme. The new work programme should indicate
well defined target groups, clearly defined objectives, as well as research tasks and performance
indicators.

3. The Panel wishes to draw attention to the fact that cross-European research in the social sciences is
at a very early stage of development. Therefore, TSER thematic networks and concerted actions
should be particularly advertised because they are adequate means to mobilise the social science
research community in Europe.
4. The Panel identified an urgent need for an overview of socio-economic research done in the
member states, in the Commission and in other specific programmes. This issue should be
discussed in the Ad Hoc Advisory Group on Co-ordination for the Targeted Socio-economic
Research Programme.

1.2.1.2 Future of Community activities in the field of socio-economic research

5. Since many of the major strategic issues facing the EU over the coming years have socio-economic
aspects, and since this area of the social sciences is so far underdeveloped in Europe, the Panel
stresses the crucial importance and pioneering role of a Community socio-economic research
programme. The Panel strongly supports the idea of a dual approach: to have a separate well
designed and coherent specific socio-economic research programme in the 5th Framework
Programme underpinning research and study activities by other specific programmes and
Commission services, and to further develop socio-economic research as parts of specific
programmes where appropriate.
6. To develop solutions for the most pressing European challenges calls for system-oriented
approaches building on multi-dimensional perspectives, where the social dimension is essential.
These issues are particularly important in the light of the tensions that are building up in the
connection of deepening and enlargement of the European Union, which have the potential to place
barriers to further realisation of the goals of the Maastricht Treaty. The social sciences have to
make important contributions both from the point of view of their broad range of special
disciplinary aspects and their ability to develop appropriate social processes and instruments for
shaping the future. Since the problems are truly European in nature they have to be dealt with
collaboratively at European level.
7. In preparing the future programme not only the definition of contents is essential but also the
design of an appropriate, well balanced set of instruments and modalities for participation:
Thematic networks, concerted actions, research projects, observatories and accompanying
measures, especially to ensure continuous communication and interaction with target audiences.

129

**13.**

8. The preparation of the future socio-economic research programme has to be realised in close cooperation with the scientific community and potential users of results.
9. In the preparation of FP5 the Commission should follow an integrative approach, ensuring the
active support and commitment of all Commission services with a specific interest in socioeconomic research.

10. The Panel recommends that appropriate means are offered to integrate into the research activities
groups from those countries from central and eastern Europe who are applying or have already
applied for EU membership.

**1.2.2** **Management and procedures**

11. To better cope with the intellectual aspects of research management at the research/policy interface
going beyond mere administrative monitoring on the level of project, project clusters and
programme, more staff with practical experience in social science research, and with capabilities
and skills in moderating interactive processes will be necessary.
12. As a consequence of the problematic personnel situation of the TSER programme, the Panel points
at an urgent need to clarify the staffing situation of the TSER programme and to take respective
steps to ensure that the programme is equipped with adequate personnel resources for its core
activities in accordance with the specific objectives and tasks of TSER and within the limits of the
indicative budget breakdown defined by the Council Decision. A personnel recruitment and
development strategy should be prepared in the very near future.
13. The Panel recommends that procedures followed in the evaluation and the final selection of
proposals are clearly described in the Information Package. The process should be organised in a
more transparent way, involving independent experts to support and advise the Commission
services and the Programme Committee in the preparation of the short list.
14. The Panel recommends that the efforts of the Commission services towards thematic clustering of
projects concerning the same strategic issues across different sub-areas and between areas should
be further developed. They will support inter-project communication, integration and co-ordination
and ensure the future impact of the programme both at the policy level and within the scientific
community.
15. The Panel particularly welcomes that Project Progress Seminars are convened by the Commission
services regularly, approximately every sixth month. Such seminars are appropriate means to
support networking with the policy makers, other Commission services and Community
programmes and initiatives, and the scientific community. In addition, also individual projects
should be encouraged to ensure interaction with possible users of their results - especially in the
policy area: integrating them in their projects or to develop other adequate means for
communication, interaction and networking.
16. The Panel feels that the TSER programme should contribute to the development of Community
RTD strategies in a well organised and co-ordinated way. Communication with CERT, CREST,
IRDAC and ESTA are of particular importance. In addition, the co-operation with the Cellule de
Prospective and the IPTS should be strengthened and further developed.
17. There is an urgent need for taking a decisive step to activate ETAN. Adequate measures have to be
taken to tap not only into the results of the TSER projects but also into socio-economic research
activities of other specific programmes and Commission services, the IPTS, and other actors in the
field both at the level of the Community and in the member states.
18. The Panel feels that exchange of information and co-operation between the TSER programme and
socio-economic research activities in other specific programmes and by other Commission services
should be strengthened. The TSER programme should have a facilitating and moderating role in
that context.

19. Co-ordination and collaboration with COST should be further developed.
20. The integration of young promising researchers and research teams in projects with wellestablished institutions is welcomed and should be further supported.
21. The Panel is concerned that a very specific issue arises in relation to the derivation, harmonisation
and dissemination of databases in the socio-economic area. In the mainstream areas of economics,
the national statistical agencies are the main bodies functioning in this area, and are very well

130

13.

resourced. However, in areas II and III of the TSER programme there is a need to generate new
quantitative and qualitative data and to harmonise them across a range of member states in a
context where national agencies are less active. Thematic networks and concerted actions should be
advertised to support that problem area. Cheap or free access to EUROSTAT data should be
provided for funded projects, networks and actions.
22. The social sciences being organised largely in very small units efficient services for better
information, advice for the preparation of proposals and support for partner search should be
further developed. In the member states the network of National Contact Points should be further
strengthened and activated where necessary. In addition, co-operation with other existing networks
should be developed.
23. The Panel proposes that at the end of that first assessment round a feedback process is organised to
draw lessons from the experiences the different panels made.

131

**13.**

**II.** **COMMISSION'S** **RESPONSE**

The Commission welcomes the Panel's report as an essential contribution towards the assessment of
Community research activities in the field of socio-economic research, and for providing detailed
recommendations regarding the future development of such activities within the 4th and 5th Framework
Programmes.

In particular, whilst wishing to recall to mind the highly dynamic nature of policy relevant research, the
Commission notes the opinion of the Panel that the rationale underpinning a specific programme in the
field of targeted socio-economic research is fundamentally sound and that the Programme's objectives
remain not just valid but also highly pertinent.

In this context, it is noted with satisfaction that in the opinion of the Panel a number of high quality
research projects and thematic networks, with the potential of generating considerable added value, have
been selected following the first call for proposals launched under the TSER Programme; and, that the
Panel considers that a number of these activities are already producing interesting and policy relevant
results.

As regards the preparation for the forthcoming third call for proposals for research actions the Panel
recommends that the work programme of the TSER Programme should be revised in order to restrict
coverage to a number of key strategic themes. The Commission also foresees the need to focus on key
issues and will consider the recommendation to revise this work programme whilst respecting also the
need to take into consideration the outcomes of the first and, the recently launched, second calls for
proposals.

Moreover, and in the context of the medium-term development of socio-economic research carried out at
the European level, the Commission welcomes the Panel's recommendation that social sciences should be
developed within the 5th Framework Programme through a well defined social sciences research
programme addressing a coherent set of social challenges facing Europe from a policy oriented
perspective, as well as being developed as an integrated part of other specific programmes where
appropriate.

Finally, with respect to the Panel's other recommendations and in particular those concerning the
operational and management aspects of the Programme, the Commission wishes to commend the Panel on
its objectivity and frankness. The Commission undertakes to address these issues and its services will
report on the measures to be taken as part of the annual monitoring exercise.

132

**13.**

**III.** **EVALUATION PANEL**

**Manfred Horvat** (Chairman)

1993 - : Director, BIT (Bureau for International Research and Technology Co-operation); Member of
INNOVATION and TMR Programme Committees; Member of ACTS Monitoring Panel 1995 and 1996;
1995 - : Vice-President, SEFI (European Society for Engineering Education); 1993-94: Visiting Fellow,
University of Oxford; 1985 - : university lecturer in Technology Assessment and Environmental Impact
Assessment (Vienna University of Technology, University of Linz); 1980 - : Director (at present on
leave), University Extension Centre, Vienna University of Technology (University-industry technology
transfer, technology assessment, continuing engineering education, management of international
programmes and projects in research and education)

**Prof Francisco Alvira-Martin**

Professor of Sociology, head of Department of methodology and research techniques, Universidad
Complutense, Madrid; he specialised in programme evaluation as a visiting scholar in North-Western
University with Prof R.B. Boruch and has written two books and more than five articles on the evaluation
research methodology; actually engaged in the evaluation of the Drug Abuse Plan of Madrid; 1995:
independent expert, panel that wrote the draft for the Evaluation Indicators System of the Spanish National
Science Plan.

**Dr John Bradley**
Economist and Research Professor, Economic and Social Research Institute, Dublin; IRL; specialised in
labour market economics and the economic processes of cohesion within the EU periphery

**Prof Pierre Desmarez**

Université libre de Bruxelles; Doctor in social sciences; Director, Centre de sociologie et d'économie
régionales; research topics: work and (un)employment, work histories, transitions from school to work

**Dr Luigi Felici**

1986 - : Director, ANCE, IT (strategic programmes in the field of urban renewal market, economic
analysis of infrastructure investments, R&D programmes in construction industries); before 1986 L. Felici
had a leading role in Confidustria's industrial R&D programmes, Mezzogiorno industrial development
strategies, environmental problems in industry, industrial policy options, external affairs and
communication); 1979 - 94: member of FAST/MONITOR Management Committee, nominated by Italian
Research Ministry

133

14.

##### **1996 FIVE-YEAR ASSESSMENT** **COOPERATION WITH THIRD COUNTRIES**

135

**14.**

**I.** **EXECUTIVE SUMMARY**

**A. Brief history of the INCO programme and scope of the** **Five-Year** **Assessment**

The Second Activity of the Fourth Framework Programme (FP4) is implemented by a specific programme
of research and technological development including demonstration (RTD) with third countries and
international organizations. This specific programme, adopted by the Council of Ministers on 23
November 1994 and known as the INCO (International Cooperation) programme, brought together for the
first time all EU RTD cooperation with third countries with the exception of the nuclear field. The INCO
programme is complemented by access to the specific programmes of the First Activity for all European
and some non-European participants in accordance with general rules established by the Community. The
cooperation now contained within the INCO programme was previously carried out through a number of
different programmes. In developing countries three different instruments of scientific and technological
(S&T) cooperation were replaced by the single programme INCO-DC. In 1991 the Community embarked
on a major expansion in RTD collaboration with the countries of Central and Eastern Europe (CEE) to
support the integration process and in 1992 launched the PECO/COPERNICUS programmes which were
later extended to the New Independent States (NIS) of the former Soviet Union. Collaboration with nonEuropean industrialized countries has developed in a variety of ways reflecting mutual interest within or
outside S&T cooperation agreements. INCO is a horizontal programme which cuts across technical fields
and incorporates the Community's linkages with COST, EUREKA and international scientific
organizations.

This five-year assessment covers the activities and management of the INCO programme and its
predecessors during the period 1991-early 1996 and incorporates the final evaluation under the Third
Framework Programme. "The four main geographical sectors covered by the INCO programme and
considered in the report are:

_Area_ _Ah_ _Cooperation with other_ _Fora_ _for European S&T Cooperation:_

_COST,_ _EUREKA, International S&T Organizations and Institutions_
_Area A2: Cooperation with the Countries of_ _Central_ _and Eastern Europe_

_and with the New Independent States of the former Soviet Union_
_Area B: Cooperation with non-European Industrialized Third Countries_
_Area C: S&T Cooperation with the Developing Countries_

The assessment panel has examined in detail the results and progress made in each main sector and
considered what useful lessons might be drawn from the Programme as a whole. The full report of the
Panel is available in English as a separate document.

**B.** **The panel's comments and findings**

1. Increasing globalisation of the economy and the spread of the Information Society are ensuring that, at
the threshold of the 21st century, international cooperation in science and technology is more intensive and
widespread than ever before. A high level of international S&T activity is required to strengthen the
science and technology basis of EU industry and to promote all the research actions necessary to fulfil
Community policies. The Panel considers that bringing together the various programmes of international
collaboration within INCO has resulted most importantly in greater coherence and visibility for EU S&T
international cooperation.

2. The Panel finds that the programmes implemented in the period under review were generally of
satisfactory relevance to the objectives laid down. A substantial number of concrete results, most with
potential for further research, have been obtained and disseminated. Some significant scientific
achievements have been recorded (Annex 11). A proper assessment of industrial exploitation would
require a longer time scale but it is not likely to be large. The burdens of management were increased by
oversubscription, communication difficulties, local conditions in some partner countries and the overriding

137

14.

requirements of accountability and quality. The formal procedures followed by the Commission were not
conducive _to_ efficiency improvements in some areas.

3. The Panel was impressed by the results that have been gained by COST and supports the existing
relationship between COST and INCO. The joint initiative in vaccine research, for example, has
facilitated long-term cooperation between the scientific research sector and industry. In the recent past
there have been notable initiatives in the chemistry domain. COST enables R&D collaboration between
scientific research bodies and industry to take place in a flexible manner as recommended in the
Commission document Coordination through Cooperation . COST also provides a nursery for projects in
specific programmes, is a useful vehicle for participants from outside the EU, and has helped to shape
elements such as ESPRIT and FLAIR in the Community's Framework Programme.

4. Because of radically different selection procedures and time-scales collaboration between EUREKA
and the Community's programmes requires exceptional efforts to achieve results but might be improved
through procedural modifications (see Recommendation 2 in Part C of this Executive Summary).

5. The Panel considers that both the COPERNICUS and INTAS programmes were essential, temporary
responses to urgent needs arising in Central and Eastern Europe (CEE) and in the New Independent States
(NIS). These programmes have helped the ongoing process of stabilising research potential in CEE
countries and in the NIS and have introduced many CEE/NIS scientists to modern methods of scientific
research in a competitive environment. The economic impact of these programmes in many CEE/NIS
countries has been very limited because of local deficiencies in infrastructure and lack of suitable
mechanisms for exploitation. High priority should now be given to increasing PHARE and TAOS
support for essential structural reforms in RTD and in the social and industrial applications of RTD. This
major challenge requires determined coordinated action and a constructive partnership with the CEE/NIS
authorities before INCO objectives in the economic, social and environmental sectors can be attained. The
potential benefits for the whole of Europe are large.

6. Progress towards participation on an equal footing in the First Activity by CEE research teams
comparable in standing with western groups should be encouraged by the EU whenever this has a realistic
basis. A separate adhesion fund outside the Framework programmes will be required to support this
participation. The transition to fully effective research partnership can be accelerated by phasing out the
COPERNICUS programme by the end of FP4.

7. The NIS, which retain a wealth of scientific expertise, particularly in Russia and Ukraine, require a
different approach. The useful experience built up by INTAS in establishing co-funded research
programmes provides a basis for continuing collaboration with important research centres, including caseby-case participation in specific programmes.

8. Despite intensive efforts by the Commission to publicise EU collaboration programmes within the CEE
countries and the NIS there is still a very low level of CEE/NIS participation in other specific programmes
and widespread lack of understanding among individual scientists of the procedures and opportunities
available. During the transition period these efforts need to be strongly reinforced through all available
information channels by very clear and unambiguous statements of eligibility for participation in different
programmes with practical details of costing and payments procedures.

9. The Panel considers that collaboration with non-European industrial countries and emerging economies
is rapidly growing in importance. It supports the views expressed in the 1995 IRDAC report that all
Community R&D programmes should be open to participation by organizations from outside Europe on a
case-by-case basis provided that European industry in general, and competitiveness in particular, can
benefit and that reciprocity and suitable IPR arrangements can be established. Emerging economies,
which are currently eligible to participate in the developing countries programme, should also be included
under the same conditions in respect of fields where they have achieved high S&T standards and
performance. The Panel agrees with the Commission's current policy of negotiating S&T cooperation

**38**

**14.**

agreements with foreign governments whenever, on the basis of wide consultation with representatives of
science and industry, there is strong evidence of mutual interest between the prospective partners.

10. The original aims of the Japan S&T Fellowship Programme have largely been accomplished and,
taking account of existing bilateral schemes, the Panel considers that this Programme should henceforth be
focused on co-funded and Japanese-funded arrangements for carefully selected candidates and host
research laboratories (university, national and industrial) of the highest quality.

11. The Panel has been impressed by the high reputation of the STD/INCO-DC programme among both
coordinators and participants deriving from the integrity of the evaluation process, the creation of
successful partnerships and the applicability of the research projects. The current practice of linking
postgraduate training (leading to a higher degree) to INCO-DC research projects should continue. The
Panel considers that the quality of peer reviewing is a most valuable asset of INCO-DC and that it is in the
strong interest of Europe to be seen to practise even-handedness.

12. While INCO's evaluation procedures are generally well regarded they are expensive in terms of
resources. The contribution of information and communications technologies to the submission and
evaluation of proposals has yet to be proven but the Panel considers that the potential benefits are large for
all INCO-administered programmes.

13. The Panel considers that globalisation is often driven by short-term economic factors and by the
prospect of access to cheap labour in developing countries. INCO has an important role in ensuring that
basic and applied sciences are not forgotten, particularly in matters that relate closely to regional issues
and problems.

14. In the opinion of the Panel the low level of direct industrial participation in the INCO programme
arises from a combination of thematic and structural causes. The IRDAC report of October 1996 contains
recommended actions which should lead to a greater involvement of industry in the cooperation
programme and to the creation of links that are favourable to innovation.

15. The breadth and complexity of the INCO programme distinguish it from most other specific
programmes and impose considerable management demands on Commission staff. Constant and close
collaboration is necessary between DG XII-B and professional staff in other directorates of DG XII and in
other DGs dealing with individual programme elements. In the view of the Panel the importance of the
INCO programme requires a strengthening of the role of the INCO Programme Committee and a
considerably higher publicity profile.

16. In considering coherence between EU and national S&T policies the Panel has come to the view that
INCO has a particular contribution to make in problems of a truly regional dimension which are not
normally tackled by bilateral programmes and for which scientific networking is of paramount importance.
INCO programmes can usefully complement national programmes by setting up European research
networks on priority topics and through interdisciplinary regional projects containing both theoretical and
applied research and requiring a balanced EU/ third country input. Coherence requires adequate
knowledge on both sides and an effective two-way flow of information. The Panel supports the INCOPOL
study which aims to collect and synthesise information on national activities in external R&D cooperation.

**C.** **The panel's recommendation**

COST. EUREKA. International S&T organizations

1. In view of the potential of COST for future expansion, its high relevance to EU policy and beneficial
interaction with the Framework Programme, an increase in resources for the COST Secretariat should be
considered and measures taken to simplify the organizational and administrative procedures as far as
possible.

139

**14.**

2. The scope for expanding cooperation between EUREKA and Community projects should be explored.
For this purpose a pilot scheme providing fast-track procedure for Community participation in support of
the research part of labelled EUREKA projects should be established with limited fund allocation and
duration. The outcome of this scheme should be used as an input to determine the extent of further support
aimed at strengthening collaboration between the Community and EUREKA.

3. Without evidence of clear added value the Panel considers that the limited activity between INCO and
international S&T organizations does not at present justify a separate category within the INCO

programme.

CEE countries and the NIS

4. Moves in CEE countries and the NIS to reform the S&T sector need to be accelerated and reinforced,
through close collaboration between INCO, PHARE/TACIS and the countries concerned, in three main
directions:

- restructuring actions aimed at concentrating S&T at a small number of centres of excellence with
advanced equipment and high-calibre scientists;

- assisting in the reorganization of S&T administration, giving emphasis to modern transparent methods
of science management, competitive grants and peer review, and complementing what is already being
done bilaterally;

- encouraging the application of S&T in society and the economy through joint efforts in tackling socioeconomic problems. Particular attention needs to be given to IPR protection as a means of stimulating
innovation.

5. The COPERNICUS programme in CEE countries and in the NIS should be phased out by the end of
FP4. During the remainder of the COPERNICUS programme much stronger focusing of calls on
priorities not covered by other specific programmes, particularly of a regional nature and established in
negotiation with R&D funding agencies, should be put into effect together with the transfer in appropriate
cases of the scientific coordinator's role to a CEE/NIS scientist. Following the end of the COPERNICUS
programme funds should be made available for more generous support for networks, conference
attendance, modernisation of equipment at selected centres, and other measures so that the amount of
contact remains at the high level sought by both eastern and western scientists.

6. The function and objectives of INTAS should be urgently reviewed so that it is able to undertake a
revised role in FP5 in accordance with the need to help restructure the S&T sector in the NIS, in
collaboration with TACIS and the countries concerned, intensify high-quality collaboration with NIS
scientists, continue co-funding arrangements and promote NIS participation in the First Activity. A joint
analysis with the Russian authorities of important concentrations of civilian research outside Moscow and
St. Petersburg would help to increase the potential for mutually beneficial collaboration. INCO should
also identify, together with TACIS, reform-minded funding agencies in the other NIS countries with a
view to setting up collaborative links along the lines of the relationship between INTAS and the RFBR
(Russian Foundation for Basic Research).

Developing countries
7. The original level of funding S&T collaboration with developing countries (DCs) should be restored in
view of the high relevance of the results of INCO-DC to current needs, the growing importance of S&T to
solving regional problems (with world-wide implications), and the need to fund a substantially greater
number of A-rated proposals and sustain the results already achieved.

8. The basic objectives of INCO-DC and the major development areas of health, agriculture and
environment should be maintained. To reduce oversubscription these areas should be more strongly
focused. The fourth area technology issues, which benefits not only industrial development but adds
value to all the cooperation programmes, should be given higher priority so that the benefits resulting from
combining technologies and from Information Society trends, in particular, can be made more widely
accessible in developing countries.

140

**14-**

9. High priority should also be given to INCO collaboration with EU programmes that support
infrastructure developments, such as ALA, MEDA and LOME, in order to raise awareness of this issue,
create stronger links between infrastructure investment and supporting RTD projects, and encourage
appropriate actions in developed countries.

10. In the light of requests from the developing countries the role of DC participants in setting priorities,
formulating and evaluating proposals, and in the scientific management of projects should be strengthened
and the possibility of entering into co-funded arrangements should be examined.

Evaluation

11. In order to make the evaluation procedures more transparent and help newcomers to Community
research programmes learn through participation in the application process, prompt notification should be
given to proposers of unsuccessful projects together with reasonable feed-back on their rating.

12. A pilot trial on the use of information and communications technologies in the submission and
evaluation of proposals should be carried out in a future call to determine whether the quality of the
preliminary stages of the evaluation process, including the necessary confidentiality, can be preserved with
a view to preparing the ground for shorter, more focused meetings in Brussels.

EU and national S&T policies

13. Continuous monitoring of RTD actions on a bilateral basis between Member States and countries
participating in PECO/COPERNICUS and INCO-DC programmes is desirable in order to adhere to
subsidiarity and avoid unnecessary duplication.

INCO Programme Committee

14. The mandate of the INCO Programme Committee should be carefully examined and revised where
necessary in order to strengthen the Committee's role in the development of a visible and effective INCO
programme and to allow time for the Committee to consider matters of a strategic and long-term nature.
The Commission, with the help of members of the INCO Programme Committee in their own countries,
should make strenuous efforts to raise the profile of INCO by widely publicising its aims and objectives
and the opportunities it provides for scientists to engage in international collaboration on relevant issues.

Administration

15. The Panel strongly recommends that the Commission give urgent consideration to the introduction of a
comprehensive Management Information System for the INCO programme capable of giving at any
moment a bird's-eye view of main trends and indicators and the status of projects. This management and
monitoring tool would facilitate coherence among all DGs actively concerned with the programme, and
enable the INCO Programme Committee and future monitoring and assessment panels to focus rapidly on
critical issues. Since some areas of the cooperation programmes appear to lack effective monitoring and
reviewing procedures for ongoing projects a system of mid-term monitoring on a sampling basis, including
independent financial auditing, should be implemented forthwith and publicised.

16. In order to streamline the administration processes within the Commission, a subject of continuing
concern to both coordinators and participants, approval for projects within a defined ceiling should be
given fast-track treatment. Consideration might be given to modifying the existing delegation and
signature procedures within the Commission consistent with the need to maintain transparency. The
Community cannot afford to show inefficiency in its own operation and at the same time urge reforms on
its DC and CEE/NIS partners.

Assessment and monitoring

17. The Panel considers that in the five-year period under review the INCO programme and its
predecessors were necessary to meet the circumstances of the time and have substantial achievements to
their credit. In a rapidly changing world there will be a continuing need for adjustments within the fouryear period of a Programme. In regard to the CEE countries and the NIS, the emerging economies and
the developing countries there will be many new challenges in the coming few years. In this situation the

141

**14.**

Panel recommends that both the monitoring and assessment exercises on the lines put forward by CREST
should remain essential components of the INCO programme.

142

**14.**

**II.** **COMMISSION'S** **REPONSE**

The Commission welcomes this independent report and appreciates the effort which had to be made by
Panel members to come to grips with such a complex programme in such a short period. The report
underlines the importance of a high level of international S&T activity to strengthen the S&T basis of EU
industry and to promote all the research actions necessary to fulfil Community policies.

The Panel recognizes the importance of the INCO Programme in providing coherence and greater
visibility to EU S&T international cooperation, and points out the particular contribution which can be
made by INCO to the solution of problems of a regional dimension which are not normally tackled by
bilateral programmes or other specific programmes.

The Commission agrees on the potential of COST for future expansion, its high relevance to EU policy
and beneficial inter-action with the Framework Programme. Any increase in resources will however need
to be carefully balanced with the needs of other parts of the Framework Programme. A High Level
Reflection Group has been constituted to consider the future role of COST in the European S&T
landscape. Maximum opportunity will be taken to simplify and stream-line the organizational and
administrative procedures, consistent with the Community's financial regulations. The detailed measures to
be worked out will draw substantially on the results of the COST evaluation now taking place, in relation
with the COST Senior Officials and the COST Ministerial Conference in May 1997.

Concerning the suggestion of a pilot scheme providing fast-track funding for Community participation in
Eureka projects, any reservation of funds specifically for EUREKA projects does not seem realistic.
However, the Commission intends to follow up the suggestion to explore means whereby urgent
EUREKA, and of course other relevant projects could nevertheless be catered for, as has been done in the
past (e.g. COSINE, JESSI).

The Commission considers that a clear point of reference responsible for international organizations is
needed to deal with the multitude of S&T issues regarding them

With respect to Cooperation with Central and Eastern European countries, the Commission agrees that,
particularly in the case of the pre-adhesion countries, full participation in Activity 1 programmes is the
ideal situation and should indeed be encouraged, it should nevertheless be recognized that participation in
these programmes through FP4 has been very low so far. Conversely, the oversubscription to the INCOCOPERNICUS programme has been in the order of 10 to 1. This is due to some extent to the different
funding regimes of Activity 1 programmes and COPERNICUS, however it also indicates that while there
is a great desire for cooperation between Eastern and Western scientists, there remains a strong
requirement for a dedicated programme. The Panel recommended that COPERNICUS be phased out with
the end of FP4 and that participation in Activity 1 be encouraged "whenever this has a realistic basis". For
most CEECs, however, such a realistic basis does not yet exist and full participation will only gradually be
achieved. The "accompanying measures" recommended in point 5 would go some way towards filling the
gaps left by COPERNICUS but probably not far enough. It should also be remembered that one of the
important roles of INCO, recognized by the Panel itself, is its contribution to the solution of problems of a
regional nature, problems which are not specifically addressed by Activity 1 programmes. In preparing for.
FP5, the Commission intends to increase the focus of the specific RTD cooperation with CEECs to
concentrate on the issues of particular relevance to these countries, as well as placing more emphasis on
accompaning measures, as suggested by the Panel.

The Commission does not share the view that the role previously played by INCO-COPERNICUS in the
NIS should be completely taken over by INTAS, including the aspect of financing the participations in
Activity 1. The role played by INTAS in S&T cooperation with the NIS is an important one, but we do
not consider that extending it further would lead to any improvement of the situation concerning
participations in Activity 1. To a certain extent, INTAS and COPERNICUS address different types of
activity. While INTAS is mainly devoted to fundamental research, COPERNICUS has a more direct

143

**14.**

approach towards application oriented and industrial research. For the management of financing
participation in Activity 1 programmes, no special agencies would be needed as this is already established
within the Commission services of the relevant programmes. In the preparation of FP5, it is intended to
look closely into the possibilities for support of Centres of Excellence in certain CEC/NIS, as
recommended by the Panel. The Commission agrees with the importance placed by the Panel on the close
cooperation necessary between PHARE/TACIS and INCO and it is intended that a joint communication
(DGIA-DGXII) on S&T cooperation with the pre-adhesion countries be produced in 1997.

The Commission notes the support of the Panel for its policies in negotiating S&T cooperation agreements
with non-European industrialized countries including Emerging Economies and shares the Panel's view
that reciprocity and suitable IPR arrangements should be established and that collaboration be implemented
through the specific programmes on a project-by-project basis. The general need to review the structure
and operation of the Japan S&T fellowship programme is acknowledged. However, the proposed phasing
out of the EU fully financed fellowships would result in eliminating the access they provide to Japanese
industrial laboratories, since the other schemes, which involve Japanese funding, are restricted to public
research organizations. The Commission welcomes the suggestion of integrating into a more general
fellowship programme the Korean and any new geographically oriented fellowship scheme, which may be
instituted with countries in Area B.

With respect to cooperation with Developing Countries (Area C), the Panel has underlined the importance
of this part of the programme and recognized that the major development areas of health, agriculture and
environment should be maintained. The Commission shares the opinion that applied research in the
technology area is important and that additional funds will be needed in order to adequately tackle this
issue. As a first step, it is intended to increase the funding available to "Additional sectors of mutual
interest" in the INCO-DC programme of FP4 from 20 MECU to 30 MECU. Specific cooperation policies
are also being developed for those Developing Countries which, due to their economic and technological
growth, should be considered as emerging economies. While the Commission appreciates the large
number of very highly rated proposals submitted in the context of INCO-DC, it recognizes the necessity to
reduce the oversubscription and thus the wasted effort on behalf of the scientific community.

Concerning the recommendations relevant to all programmes, the Commission agrees that it is necessary
to constantly make efforts to reduce perceived "procedural inefficiencies" leading to long delays between
submission of proposal and the first transfer of funds to the successful applicants and to ensure that
unsuccessful applicants are informed rapidly of their situation. Continuous attempts are being made to
ameliorate the internal Commission administrative procedures while remaining consistent with
Commission's duty of public accountability.

The electronic submission of proposals has already been tried by some programmes in the past. The
possibilities could be examined again in the light of updated electronic methodologies. The facilities
available in relevant countries would need to be investigated and to be taken into account.

The Commission fully agrees with the recommendation that an integrated Management System be
introduced. Although the suggestion is relevant to all programmes, it is particularly so in the case of
INCO, where several different DGs are involved in its management. It is intended to improve this situation
as far as possible over the remainder of FP4, and particularly to have a system in place before the
commencement of FP5.

Concerning the role and mandate of the INCO Programme Committee, it is governed by the Comitology
Decision of 1987 and Article 130i of the Maastricht Treaty.

144

**14.**

**III.** **EVALUATION PANEL**

**Mr Luis Avelar**

Director, Portugal Telecom

**Prof.** **Niels Ole Bernsen**

Center for Cognitive Science
Roskilde University
Denmark

**Dr.Stein W Bie**

Director, Research, Extension and Training Division,
Sustainable Development Department
Food and Agriculture Organization of the United Nations

**Mr Christian Dambrine**

Association Nationale de la Recherche Technique, France
Member of the Bureau, European S&T Assembly

**Prof.** **Dervilla** **Donnelly**
University College Dublin, Ireland
Vice-President, European S&T Assembly

**Dr Désirée** **Edmar**

Director, Naturhistoriska Riksmuseet
Swedish Museum of Natural History, Stockholm

**Mr Terence Garrett**

Formerly Department of Trade and Industry
and the Royal Society, UK

**Dr Werner Menden**

Formerly Bundersministerium fur Forschung und
Technologie, Germany
Member of the 1995 Monitoring Panel for the INCO Programme

**Dr.Alberto Silvani**

Consiglio Nazionale delle Ricerce
Italy

145

15.

##### **1996 FIVE-YEAR ASSESSMENT** **DISSEMINATION AND OPTIMIZATION OF RESULTS** **(INNOVATION)**

147

**15.**

**L** **EXECUTIVE SUMMARY**

1. Among European decision-makers, there is general agreement that frequent and systematic
innovation is a main source of new jobs, high productivity, and, in the end, of wealth creation. But
innovation needs a favourable environment. This means management skills, circulation of
knowledge across countries and sectors, flexible product markets, and market-oriented RTD. It also
means standards and regulations that promote innovation; a tax policy conducive to innovation;
conditions favouring the development of capital markets for financing innovation; a European patent
policy that facilitates access to intellectual property rights and defends those property rights more
effectively worldwide. Finally, much more than is now the case, research institutions and industry
should work together to meet the needs expressed on the market by users and consumers.

2. INNOVATION, the specific programme corresponding to the Third Activity of Framework
Programme IV, deals with most of this. INNOVATION became operational following the Council
Decision 94/917/EC of 15 December 1994. The programme has a budget of 312 MECU. It carries
on activities started under SPRINT and VALUE II, and one action line from THERMIE. Its basic
mission is to strengthen the innovative capacity of European industry. The programme must also
contribute to economic and social cohesion. There is a special emphasis on SMEs and technology
diffusion and circulation. All the programme must be conducted under the principle of subsidiarity:
it must complement, but not be a substitute for, national and regional activities in similar domains.
3. The INNOVATION programme is evolutionary and experimental. It is particularly important,
therefore, that it be regularly refocused by proper monitoring and evaluation based on verifiable
objectives. The time and resources available for this report, however, have dictated that it be more a
review on the basis of relevant documentation, some interviews, and our general knowledge of the
area, rather than an evaluation with strong empirical backing. The Commission should adopt
measures to ensure more effective evaluation in future.

4. The Terms of Reference for this assessment included five questions. We were asked to judge the
relevance, effectiveness, and cost efficiency of the initiatives carried out within the domains of the
INNOVATION programme in the years 1991 to 1995. Moreover, having in view the socioeconomic conditions expected pver the five years to come, we had to make recommendations about
the future orientation and management focus of the programme. The assessment had to take into
account the final evaluations of VALUE II, which was part of Framework Programme III, and of
SPRINT, which was outside the Framework Programme.
5. **Our overall impression is positive.** INNOVATION and its precursors SPRINT and VALUE II
have addressed issues relevant to the Community's socio-economic needs. These programmes have
increased the understanding of the innovation processes and policies within the Commission and
among national and regional actors. The merger of SPRINT and VALUE II was a necessary step to
rationalize previously separated streams of action. There are some priorities to reassess and some
shortcomings to fix, but these problems are not paramount.
6. **In particular, INNOVATION and its precursors have:**

    - Set up infrastructures to disseminate information on the Community RTD activities;

    - Learned and disseminated useful lessons on the innovation processes in SMEs;

   - Promoted horizontal actions within the Commission to make RTD and other Community policies
more innovation oriented. Regional actions are the most successful example;

   - Created a seed for a European innovation policy. The Green Paper on Innovation should get much
of the credit for this;

    - Recognized that action at Community level should complement national and regional activities, not
be a substitute for these.

7. **We believe that innovation policy should have an enhanced role in EU activities.** We emphasize
in our report the importance of local connections and even personal contacts in diffusing innovative
ideas. We believe, however, that there is a place for Community action, provided that excessive
centralization is avoided. The action plan based on the Green Paper on innovation should lead to a
European innovation policy with a distinct and strong role among the other EU policies and among
national and regional innovation policies. This policy should be supported by new organizational
arrangements within the Commission.

149

**15.**

**8.** **In the light of this, it would be right to give INNOVATION a strategic and more influential**
**role.** INNOVATION has been hampered by having too few resources and too little authority. It has
nevertheless made progress in the direction of a European innovation policy. It therefore deserves to
be enhanced.

9. **INNOVATION should be a wide-ranging and flexible entity within the Framework**
**Programme.** There is a complex relationship between RTD and innovation. Each contributes to the
other, but innovation is a wider concept and has other roots, besides those in RTD. On balance, we
feel it is right to keep INNOVATION within the Framework Programme. The Framework
Programme, however, must not unduly constrain INNOVATION from its wider activities. This
means developing a complex of horizontal interactions and may need a rather more open structure
than is appropriate for the RTD programmes. As we mentioned before, the programme must be able
to experiment and evolve. This, in turn, means there must be well defined procedures for initiating
and terminating activities, i.e. 'entry and exit mechanisms'. This also implies that the programme
must be able to respond to needs that change rapidly. Flexibility should be increased.
10. **We would** **like** **to see INNOVATION and its staff acting as a 'ginger group' within the**
**Commission as well as outside it.** A 'ginger group' is a small group of people whose role is to
provoke and stimulate activity, to encourage new thinking and hence to enliven a larger group. This
is precisely the part we think those involved with the programme could and should play. By
disseminating ideas, new experience, and the results of experimentation, they would provoke and
stimulate innovative approaches. This would give the programme staff something of the nature of a
think-tank for pursuing the aims of the Green Paper. They would need to maintain a clear view of
how innovation works, provide a strong element of reflection and experimentation, and have the
main objective of promoting innovation inside and outside the Commission.
11. **We recommend strengthening the programme's coordination with other EU policies and DGs.**
Interfacing with, and contributing to the policies of, other DGs would be a vital element of this
'ginger group' role. In particular, the programme should be linked with the assessment and
implementation of activities under Community policies other than RTD, such as the industrial,
SME, and regional policies: all this to help innovation, as well as technology transfer and diffusion.
These matters require a committed advocate, but also proper administrative arrangements.
12. **It** is **particularly important to enhance the interaction between the Third Activity and the RTD**
**programmes of the First Activity.** In our view, the INNOVATION programme's mandate does not
distinguish adequately between exploiting the results of specific RTD projects and promoting
innovative thinking and practices more widely. Within Community-supported research, we think
there should be greater attention to exploitation, but this should be the responsibility, primarily, of
the RTD programme involved. INNOVATION, however, should not be a mere downstream service
available to the RTD programmes on request. It should provide them with guidelines for
dissemination and optimization and play an interactive role of monitor and mentor throughout their
whole life. In general, it should increase the orientation of these programmes towards innovation.
INNOVATION has launched a number of specific actions to ensure a coordinated approach to
dissemination and optimization. When fully operational, these initiatives could make good progress
in the direction of effective coordination. INNOVATION should have the authority and institutional
instruments necessary to ensure that this occurs.
13. **The programme will need to be clearer about its priorities. It will also need a simpler**
**structure. All this to give a sharper focus on some issues, increase flexibility, and improve**
**internal coordination. We draw attention to the following points:**
13.1 The European Innovation Monitoring System (EIMS) should be a particularly important part of the
INNOVATION programme facilitating, as it already does, the integration of Community activity
with that of member states and other organizations. It could play an important role within the
'ginger group' activities we advocated above for the programme.
13.2 The programme should expand and intensify its regional actions. In particular, using existing
initiatives such as the Regional Innovation Strategies or new, more specific, instruments, the
programme should help eligible regions to make efficient use of structural funds in domains affected
by technological innovation and should encourage SMEs in these regions to participate in
cooperative innovation projects.

150

15.

13.3 The INNOVATION Relay Centres (IRCs) are in the best position to integrate action at the local and
European level. The IRCs should strengthen this as their prime role within the programme. The
IRCs should work on promoting innovation in their local environments rather than concentrate on
disseminating information about Community RTD. Whenever useful, they should give a European
orientation to the demand for innovation and innovation financing of local firms. They could also
provide valuable services to other parts of the programme, such as the regional actions. To do this
they need to be aware of all aspects of innovative thinking and to be very close to their local
economies, building on, but not duplicating, other locally based activities. They also need to be
close to the Commission and to other IRCs so that they can operate as an effective network.
13.4 Creating networks is in our view an important role for innovation policy at European level. Through
these, information and ideas will flow, enabling cross-fertilization of regional and sectoral
experience to occur and stimulating competitive advances. Such networks will sometimes be
electronic, sometimes paper-based, and often working through personal contact. They may be either
permanent, i.e. infrastructural, elements of policy, or experimental initiatives meant to continue on
their own. Within the INNOVATION programme, networking initiatives other than the IRCs should
be experimental. They should be designed to become in due course financially independent from the
Commission. These initiatives could add a European dimension to networks that already exist or
start cooperation in some new areas where the Commission has competence.

13.5 We attach particular importance to stimulating a financial environment favouring the diffusion of
new technologies. So far, however, the INNOVATION programme has made only limited
contributions to thinking in the field. It should aim at a larger role. Europe has been too slow in
developing systems for financing innovative activity. These need to be developed in a coordinated
way to integrate actions at regional, national, and Community level.
13.6 The programme should retain the important information-providing role which it discharges via
CORDIS (preferably, further enhanced). It should give more attention to understanding, in depth,
the needs of users of the service so that the innovation message is broadcast to a receptive audience.
13.7 We strongly recommend additional effort and a more coherent strategy on patents. This means
primarily strengthening the present role of support to the RTD programmes. But the development of
provisions for intellectual property rights is also a strategic issue for a European innovation policy.
The current situation is far from satisfactory. Measures in this area should be intensified in future.
13.8 The purpose of projects within the INNOVATION programme should be to demonstrate and
disseminate innovative approaches and ways in which technology can enhance innovative attitudes.
It should not be, only, to exploit specific results. These projects should, primarily, serve as
showcases for displaying innovative thinking and practices generally. They should aim, rather than
at specific technologies, at the wider objectives of a European innovation policy such as an easier
access to product markets for new technology-based firms, the increase of the technology absortion
capabilities in traditional sectors, a more frequent transfer of technologies between sectors and
countries (not only within the Community, but also across its borders), the stimulation of a 'culture
to innovate' throughout Europe.

13.9 We believe that the transfer and validation projects should be merged, the OPET network should be
transferred to THERMIE, and two of the present action lines, innovation management techniques
and public awareness of research and technology, should not be individual elements of a future
programme, although their specific goals may be important. The awareness activities could become
a strand of the EIMS and be redirected to a coherent promotional strategy of the Community
initiatives in research, technology, and innovation. The innovation management initiatives could be
shared out: the EIMS would assess new techniques, whereas the IRCs would promote state-of-theart practices.
14. **The previous recommendations may be summarized in the six points below:**

    - Innovation policy should have an enhanced role in EU activities.

   - INNOVATION should continue, be significantly enhanced, and become a wide-ranging and
flexible entity within the Framework Programme. It should provide a 'ginger group' to stimulate,
promote, and champion innovation within the Commission as well as outside it.

   - There should be greater attention to innovation within the RTD programmes.

151

15.

- INNOVATION should assist the dissemination and optimization activities of the individual RTD
programmes. It should have the opportunity to intervene where these activities appear to be
inadequate. In general, it should increase the orientation of the RTD programmes towards
innovation. Proper administrative arrangements are needed to ensure all this.

- The coordination with other EU policies and DGs should be strengthened. In particular, the
programme should be linked with the assessment and implementation of activities under Community
policies other than RTD, such as the industrial, SME, and regional policies: all this to help
innovation, as well as technology transfer and diffusion.

- There should be greater attention to ordering the priorities of the programme's individual action
lines so as to give a sharper focus on the issues highlighted in the previous points. In future the
programme would consist of eight major elements.

152

**15.**

**II.** **COMMISSION'S** **RESPONSE**

The Commission fully supports the emphasis in the recommendations on the promotion of innovation, both
in-and outside the RTD Framework Programme.

As a matter of fact the Commission adopted on 20 November 1996, following the wide ranging public
debate stimulated by the Green Paper on Innovation, the First Action Plan for Innovation in Europe. This
Plan considers the ability to innovate as a necessity to maintain competitiveness and a precondition for the
future creation of jobs in Europe.

It identifies three areas for action : to foster a genuine innovation culture, to establish a legal, regulatory
and financial framework conducive to innovation, to better articulate research and innovation. Each of
those areas includes actions which belong to the realm of science, research and technology policies as well
as actions which typically belong to other policies such as industrial -, enterprise -, and regional policies.

The Commission also agrees with the recommendations that there should be greater attention to innovation
within the RTD programmes and that the INNOVATION programme could usefully guide and assist the
specific programmes in these efforts. In the light of the experience of the Interprogramme Group on
Dissemination and Optimization, a specific mandate will be considered for the INNOVATION programme
providing the basis and instruments necessary to ensure that this mentoring and monitoring effectively

occurs.

The same applies as regards the recommendation on the coordination with other EU policies with a view
to promote innovation, technology transfer and technology diffusion.

Thirdly, the Commission welcomes the proposed re-structuring of the activities covered by the programme
and the suggested reduction in the number of action lines. In this context it wants to point out that the

latest call for proposals for Technology Transfer and Technology Validation Projects 1 already has come a
long way towards integrating both types of projects as indicated by one of the recommendations.

Finally the Commission has taken notice of the recommendations that the INNOVATION programme be
clearer about its priorities : exploiting the results of specific RTD projects versus promoting innovative
thinking and practices more widely. In line with the orientations adopted by the Action Plan for Innovation
the latter is to emerge as one of the key priorities for future operations

OJN°C221, 17.09.96, p. 20.

153

**15.**

**III.** **EVALUATION PANEL**

**Luis A. Oro** **Giral** (E), Chairman
Professor, Department of Inorganic Chemistry
University of Zaragoza

**Nicos M. Christodoulakis** (GR)
Professor, Department of International Economics and European Studies
Athens University of Economics and Business

**Heinrich** **Hôfer** (D)
Head, Department of Technology and Innovation Policy
Federation of German Industries (BDI, Bundesverband der Deutschen Industrie)

**Sean McCarthy** (IRL)
Managing Director, Hyperion Energy Systems Ltd

**Erkki** **Ormala** (SF)
Secretary, Science and Technology Policy Council of Finland
Ministry of Trade and Industry

**Philip V. Hills** (UK)
Professorial Fellow, PREST (Policy Research in Engineering, Science, and Technology)
The University of Manchester

**Giorgio Tuninetti** (I), Rapporteur
Assistant to the First Vice-President, Business Development
Corporate Staff, Finmeccanica SpA

154

16.

##### **1996 FIVE-YEAR ASSESSMENT** **STIMULATION OF TRAINING AND MOBILITY** **OF THE RESEARCHERS**

155

**16.**

**I.** **EXECUTIVE SUMMARY**

This is the Executive Summary [1] of the Five Year Assessment of the Human Capital & Mobility (HCM)
and its successor the Training & Mobility of Researchers (TMR) Programmes. The Assessment is
undertaken as a precursor to the adoption of the TMR Programme under the Fifth Framework
Programme. It also incorporates the Final Evaluation of the HCM Programme.

The recommendations made in this Report are essential to the good development of the TMR Programme
and have the total and unanimous support of the Assessment Panel. It is the Panel's explicit wish to see
these recommendations fully implemented.

Europe, as it enters the 21 [st] century, faces a number of major challenges. Its industrial competitiveness
must be improved and barriers to the flow and development of information and knowledge broken down.
Ecological and environmental problems must be overcome. The major social concerns of migration and
racism, health and lifestyles, social exclusion and governance must be faced. The Panel concludes that
Europe will be better positioned to face theses challenges, if its scientific and technological research
community is ready to cooperate across discipline, across culture and across regional and national
boundaries and that a training and mobility programme has a substantial contribution to make in
developing this cooperation.

- **The TMR Programme is an essential element in facing these challenges and should be continued**
**under the Fifth Framework Programme.**

However, any programme must have the capability of transcending purely EU concerns to ensure research
encompasses the Global dimension of industrial competitiveness and of ecologically sustainable
development and, in particular, plays a full part in the development and stabilization of Central and
Eastern Europe. To respond fully to these challenges, some changes are needed in what the Panel
considers to be a most successful and well run Programme.

- **The Programme needs to move to working with all research disciplines relevant to these**
**challenges and developing in them a full European and Global dimension.**

The Programme must also work closely with European research community itself, where it has earned
great respect for its emphasis on scientific excellence and a "bottom-up" approach. The latter permits
dynamic new areas outside of the EU specific Programmes to be supported and corrects for the
misalignment or inflexibility of highly targeted programmes. It is in a sense an insurance policy.
However, it does cause major problems in over-subscription in parts of the Programme and confusion in
Programme objectives : is it an insurance policy for the research not covered by specific Programmes or is
it a training and mobility programme ?

The Panel recommends that :

- **All 5FP Programmes should have a section for "bottom-up" research - their insurance policy.**
**The TMR should remain bottom-up but concentrate on training and mobility issues.**

Turning to the individual Programme Activities :

- **The Fellowships Activity** should be developed as three basic activities. The Panel recommends that :

1. The Marie Curie Fellowship should be developed explicitly for the very ablest of Europe's
researchers with a proven research record.

^ h i s Summary is published on a stand-alone basis in all Union languages (Document 1) and as part
of the full report in English only (Document 2).

157

16.

2. Industrial Host Fellowships should provide support specifically for the training researchers in
industry.
3. Structural Cohesion Fellowships should provide specifically for the development of high level
research teams in Less Favoured Regions.

- **The Research Networks Activity** should increase the success rate and value for money of networks
by:

1. Increasing the flexibility of network structures funded, possibly moving away from the average
funding per node basis.
2. Providing a maximum of 50% funding for projects,
3. Increasing quality of networks fellows by open, international advertisement of positions,
4. Encouraging interdisciplinary proposals and ensuring their equitable evaluation structure.

- **The Large Scale Facilities Activity** should :

1. Increase the cluster development and co-ordination elements of the Activity.

These have a high EU value added.
2. Broaden their definition to being "Large Scale Intellectual Centres of Excellence".
3. Avoid becoming the platform for the planning of news LSFs. This might become damaging to
the main objectives of the Activity.
4. Be reviewed for repositioning within the Framework Programme.

- **The Accompanying Measures Activity** : There is a major need for a "Strategy & Follow-up Unit" to
support the Activities of the TMR Programme. This Unit will have many tasks essential to overall
Programme development including developing :

      - An overall strategic and analytic capacity to support the Programme,

      - A working model of the European industrial and academic research community needed for
Programme planning and evaluation,

     - A follow-up capacity which will coordinate and analyse the impact of the TMR
Programme,

      - Associated feedback mechanisms to identify shortcomings and potential improvements,

      - A support platform to work with Member States on trends in research training, mobility and
related issues.

158

**II.** **COMMISSION'S** **RESPONSE**

1. The Commission welcomes the principal conclusion of the Panel that the promotion of the training
and mobility of researchers is an essential element of the Fourth Framework Programme and should be
continued under the Fifth Framework Programme. The Commission shares the Panel's view that the
current Training and Mobility of Researchers (TMR) Programme is a most successful and well run
Programme.

2. The Panel has made a number of specific recommendations to improve still further the impact and
effectiveness of these training and mobility actions, in particular with a view to the Fifth Framework
Programme. These recommendations will be studied carefully to see how they can best be implemented,
in particular:

  - focusing the Marie Curie Fellowship scheme on the very ablest of Europe's young researchers
with a proven research experience;

  - introducing Industry Host Fellowships to provide support specifically for the training of
researchers in industry, as well as Regional Host Fellowships to improve the development of
research and research training in the less-favoured regions of the Community;

  - finding mechanisms to increase the success rate and value for money of Research Training
Networks;

  - introducing Infrastructure Networks as a means of promoting self-coordination among Europe's
research infrastructure;

  - organising a Strategy and Follow-up Group to provide strategic and analytic support for the

activities.

3. The Commission shares the Panel's view that these actions should remain "bottom-up", open to all
fields of scientific research, and with projects selected on the basis of scientific excellence.

159

**16.**

**III.** **EVALUATION PANEL**

**Professor Daniel Thomas** (Chairman)
Professor at the University of Technology of Compiègne (France). Since 1970, he has held responsibilities
at a national and European level. He heads the enzyme technology laboratory belonging to the CNRS. He
has published over 300 scientific papers in international journals. For several years, he was Director of the
French Biotechnology Programme. At a European level, he initiated the Biotechnology activities in 1978
and has chaired many E.C. committees.

**Professor Joannes (Jan) Beenakker**
Emeritus Professor of Physics at Leiden University. Before his retirement, he headed the molecular
physics group of the Department of Physics at Leiden University. His main field of research over the past
decade has been the study of nonequilibrium behaviour of gases of rotating molecules. He has been guest
Professor of molecular physics at the Catholic University of Leuven. From 1985 until 1991 he was Rector
Magnificus of Leiden University. After his retirement, he served as Chairman of F.O.M. (the physics
section of the Dutch Science Foundation NWO), from 1991 until 1994. He is a member of the KNAW,
the Royal Dutch Academy of Science and of the Academia Europea.

Mr. **Tom** Casey (Rapporteur)
Director, human Resources, at the CIRCA Group Europe Ltd. and a lecturer in Technology & Innovation
Policy in University College Dublin. He was educated at Imperial College, University of Manchester and
Université de Paris-Dauphine. He has worked in the OECD, National Board for Science & Technology,
National Enterprise Agency in Dublin.

**Professor Peter** **Hackl**

Professor of Statistics at the University of Economics and Business Administration, Vienna, Austria. His
scientific interests concern the development and applications of statistical methods for time series analysis
and for the analysis of economic data e.g. diagnostics for economic and econometric models and methods
for process control and for quality improvement. He is author of several books and some 70 articles in
refereed international journals. Since 1995, Professor Hackl has been President of the Austrian Statistical
Society.

**Dr. Geoff Robinson**

Director of Technology for IBM United Kingdom. He has spent most of his career in Research and
Development for IBM. From 1992 to 1994 he was Chief Scientific Advisor on Science and Technology at
the UK Department of Trade and Industry. He is a Fellow of the Royal Academy of Engineering and was
President of the British Computer Society 1995-96.

**Professor Alexandre Quintanilha**
Holds a PhD is theoretical Physics and is Professor of Biophysics and Dean of the Biomedical Institute of
the University of Porto. From 1972 to 1990, Professor Quintanilha was Associate Professor of Physiology
at the University of California at Berkeley and Director for Environmental Studies at the Lawrence
Berkeley Laboratory. His area of scientific research is molecular markers and mechanisms of adaptation
under stress in humans.

160

**17.**

##### **1996 FIVE-YEAR ASSESSMENT** **NUCLEAR FISSION SAFETY**

161

**17.**

**T.** **EXECUTIVE SUMMARY**

**The main impact of the programme** has been, and continues to be, the provision of a common scientific
basis for:

the safe operation of nuclear reactors,
their effective decommissioning,
the development of agreed means for the safe management of radioactive waste, and
the protection of the European population and environment from possible adverse effects of
radiation, and in particular the past, present or future use of nuclear power and the uses of radiation
in medicine.

**In reviewing the achievements of the Third Framework Programme (FP3)** we have concentrated our
detailed attention on the Reactor Safety and Waste Management areas, since the Radiological Protection
and Decommissioning areas have been subjected to independent evaluation during this five-year period.

In **reactor** safety, new understanding has been developed of phenomena which may occur in a severe
accident. Progress was made in the area of core degradation, but uncertainties remain in the so-called
late phase. Progress was achieved in calculating distribution and combustion of hydrogen, and basic
knowledge was obtained in the area of molten fuel-coolant interaction. Several interesting core retention
devices were developed. Fission product behaviour was studied effectively. Accident management
support was initiated to be available to help in recovery following a severe accident.

In **radioactive waste management,** there has been cost-effective use of expensive underground
laboratories both to develop confidence in safety performance assessment for the selection of future
suitable sites which will meet national criteria, and to understand mechanisms of migration of
radionuclides, many of which are relevant also to other toxic substances. There has been useful
innovation in attempting to reduce the volume of various wastes, the content of long-lived alpha-emitting
materials to be disposed of in repositories, and in the development of waste forms and containers which
are stable in the long term. A 'European Network for Quality Checking of Waste Packages' has been
successful in providing harmonization of quality assurance for radioactive waste products which has been
of help to national regulatory and licensing organizations.

Research in **decommissioning** has decreased as the technology has matured, and the publication of the
Handbook on Decommissioning of Nuclear Installations has proved an excellent means of disseminating
results and of stimulating their practical application.

In **radiation protection,** the major achievement has been in providing the scientific basis for the
revision of the EC Basic Safety Standards Directive, which was adopted in May 1996. Fundamental
advances in understanding DNA repair after radiation damage, the initial events in radiation
carcinogenesis, and in epidemiology have come from the programme. It has also helped to evaluate
patient benefit from conventional and newly-introduced radiological techniques and contributed to the
revision of the Medical Exposure Directive. Specific achievements in retrospective radiation dosimetry
have also resulted from the programme. The socioeconomic value of the programme can be evaluated in
terms of life prolongation and improved quality of life through better medical care, or averted medical
cost due to avoidance of adverse radiation effects.

The analysis of the ecological consequences of the Chernobyl accident has thrown into relief which
remedial actions are beneficial and which are less so. A new approach to radioecological studies has led
to the development of models which may contribute to a comprehensive environmental strategy for
response to any future accident. Finally, much has been achieved in developing and disseminating within
the EU computer programmes for risk assessment and decision support following a nuclear emergency.

163

**17.**

The **transition from FP3 to the Fourth Framework Programme (FP4),** with a smaller number of welltargeted research areas, increased numbers of partners in each project, increased multi-national
cooperation, increased participation by industry and by small and medium-sized enterprises has led to
improved management of what was already an effective and productive European research effort. In
several areas of this programme, the work of European organisations represents the leading edge of world
science, and these areas do represent real opportunities for added economic value to European industry.

An important outcome of this EU-wide research and training programme is the maintaining of scientific
and technical competence even in those EU member states where such knowledge is not provided as a byproduct of operating nuclear power plants. The continuing safety of populations can be assured only if
sufficient knowledge is available in all countries. This may require that a new generation of suitably
qualified scientists emerges to replace those who are retiring. Active participation in future well-targeted
EU research programmes is an excellent means for fostering a new generation of specialists.

The common scientific base, to which this research programme has made significant contributions,
allows EU member states to make their own policy decisions, but helps to ensure that the logic by which
those decisions evolve is apparent.

The major method of communication of results obtained in this programme is _via_ publication in refereed
scientific journals, but publications by the Commission have made a useful contribution in dissemination of
information about the programme. European and wider international scientific audiences have been
reached successfully by the numerous symposia, workshops and conferences organised within the
programme. However, the use of the results of this programme by other Directorates of the Commisson,
_e.g._ in setting standards, in improving public confidence, and in assistance to countries in central and
eastern Europe, has been less than we would have hoped.

The **development of FP4** represents a significant increase in detailed specification by the Commission
of the types of studies which would be supported, although the programme evolved from wide discussions
to allow input from individual scientists. It is important that a balance is maintained to ensure that
opportunity exists for innovative scientific groups in member states to obtain EU funding for work in new
or unorthodox areas where the chance of success may be low but the potential value of success is
disproportionately high.

**FP4** provides an excellent base **for defining needs for the Fifth Framework Programme (FP5).**
However, there will be advantage in reviewing objectives and time schedules in view of end-user needs.
End-users of each of the programmes should be identified, and links established between the research
teams and the users. Several results from the programme can be tailored so that they are used by small
and medium-sized enterprises, _e.g._ in radiation control, reactor safety, waste management and emergency
preparedness.

By concentrating further research on risk relevant items, extensive knowledge about severe accidents
should be available by the end of FP5, providing a common scientific basis of understanding at the
European level about conceivable reactor accidents and corrective measures. This knowledge can be
utilized in appropriate accident management procedures and, where applicable, design features.

Underlying knowledge for waste repositories should improve up to the time of actual disposal. The
main task is now to show that all the necessary studies and experimental investigations are carried out to
achieve, to the required degree, the safe performance of the entire waste management system - especially
the geological disposal. In this context, the long term tasks require further effort, not least because of the
wide range of potential geological formations in the EU member states.

We consider that no further research work will be required on decommissioning, since the current
programme in FP4 will lead to demonstration of a comprehensive range of successful techniques on a
wide range of reactor stations and other nuclear installations.

164

17.

Study of human protection against radiation resulting from nuclear power activities requires continued
research. This area is important in demonstrating improving quality of life for European populations, and
relieving them of some of their fears. It is an area of scientific excellence and exceptionally broad
European collaboration, where the activities of the Commission can guide and ensure enhanced output of
national research. Even if some aspects of the programme in the radiation protection area are of no direct
relevance to nuclear safety, they are of great significance to the protection of the patient in medicine and
should be continued.

We consider that the area 'Mastering Events of the Past' can be merged into the research areas
mentioned above.

Detailed **recommendations** at the end of the main text include suggestions for further improving the
Commission's administrative procedures, so as to leave more time for the contractors to concentrate on
their research.

165

**17.**

**II.** **COMMISSION'S** **R E S P O N S E**

The Commission acknowledges the work of the independent evaluation experts who were confronted with
the task to evaluate for the first time the Euratom research and training areas on Reactor Safety,
Decommissioning and Radioactive Waste Management and Disposal together with the Radiation
Protection Research.

Since the Radiation Protection Research contains some subject fields, such as radiation from natural
sources and medical use of radionuclides and ionising radiation, which are not relevant for all the research
areas of the SP Nuclear Fission Safety, the evaluation panel had the difficult task to situate the various
research achievements of the past as well as to emphasise research objectives for the future.

It is in line with the Commission's strategy for planning future research that a more coherent approach
should be followed and that the protection of the population and protection of the environment will remain
a main objective of the NFS research. A further important aim will be to stimulate the conservation of
expertise and industrial activities which assure European competitivity in the nuclear field. Equally
important, when defining new research objectives, a greater emphasis will have to be put on the end-user
of the research results and on the establishment of 'closure criteria' in order to set more precise time limits
to the research projects.

The Commission concurs with the specific conclusions that research on severe reactor accidents should be
continued and that innovative European reactor designs should be supported in the future. Furthermore the
recommendation will be considered that safe performance of the waste management and in particular
geological disposal will be of special importance for this programme.

The Commission shares the panel's opinion that an EU programme of radiation protection research must
continue to ensure safe application of ionizing radiation in the different fields, as far the public, the
workers and the environment are concerned. It will thus help to establish an objective framework to
prevent distorted public perception of radiation risk from denying to Member States the undoubted benefits
of wider uses of radiation in medicine and industry, and not just in nuclear energy.

The Commission underlines that one particular role of its research programme in NFS is to contribute to
establish a baseline for apparent logic in the policy decision making of the EU Member States. The
continuing European programme will help to assure complementarity and quality of work across the
Member States.

The Commission takes up the panel's view point that more attention will be paid to cooperation and
management aspects, in order to improve the efficacy of the EC's effort. This includes that scientific
excellence and management skills will remain the yardsticks for future selection of research priorities.
The EC's research programme will also be conceived to attract young scientists to these fields of
research where European competence risks to loose territory on the global scale.

166

**17.**

**III.** **EVALUATION PANEL**

**Mr BENNETT, B.G**

UNSCEAR Vienna

Secretary/Committee on the effects of atomic radiatons

**Mr ESCHRICH, H.**

retired

Nuclear reactor specialist

Mr **FEATES,** F.
University of Manchester
Professor/environmental radiation chemistry

**Mr GUILLAUMONT, R.**

CNRS

Professor, Institut de Physique Nucléaire

Mr **MARCUS,** F.R.
NKS (Nordic Nuclear Safety Research)
Expert in nuclear safety research

**Mr** **VAYSSIER,** **G.**
Nuclear Safety Consultancy
Consultant

**Mr WAMBERSIE, A.**
Université Catholique Louvain (UCL)
Professor, faculté de médecine, radiobiologie et radioprotection

**Mr BERRY, R.J.**

Consultant

Nuclear Safety Consultant

167

**18.**

##### **1996 FIVE-YEAR ASSESSMENT** **CONTROLLED THERMONUCLEAR FUSION**

169

I. EXECUTIVE SUMMARY

**Recommendations**

_1._ Fusion is one of the few energy sources which might make a significant contribution to satisfy the
growing need for electricity from the middle of the 21st century onward. Taking into account
intrinsic safety aspects, potential environmental advantages and the wide availability of fuel, it is
important for Europe to have this option open. The long-term and the wide scope of fusion
development justify, in application of the principle of subsidiarity, the continuation of the direct
involvement of the European Union in this R&D area, and of the full integration of the efforts of the
Member States (and Switzerland) in a European Fusion Programme.

2. The Board confirms the validity of the long-term R&D strategy recommended by previous panels and
endorsed in the 1994 Council Decision: on the path to the Demonstration reactor (DEMO), only one
large device is needed i.e. a tokamak experimental reactor (ITER).

_3._ The Board, welcoming the strong efforts developed by the Programme to focus on ITER most of the
scientific, technological and industrial activities, recommends that this effort be continued and further
steps be taken to increase the involvement of industry in the Programme.

_4._ Fusion R&D has now reached a stage where it is scientifically and technically possible to proceed
with the construction of the first experimental reactor, and this is the only realistic way forward.
Starting the construction of ITER is therefore recommended as the first priority of the Community
Fusion Programme under the Fifth Framework Programme.

5. ITER should be built in Europe, as this would maintain Europe's position as world leader in fusion
and would be of great advantage to European industry and laboratories. This would, however,
require an average increase of at least 50% in the Community funding for Fusion in the first decade
of the next century, accompanied by the phasing out of JET, a redirection of the Community funding
of the Associations, and a substantial contribution from the host country. For the Fifth Framework
Programme the additional financial requirement would be limited to the order of 200 MECU.

6. If it turned out not to be possible to have ITER in Europe, the firm recommendation of the Board is
to maintain a strong participation in ITER as the first priority of the European Fusion Programme.

7. The Board recommends that, with regard to the construction of ITER : particular attention be
devoted to the organisation of System Engineering and to the consequent optimisation of the
industrial participation; the contributions by the partners be provided mainly in kind; and a substantial
fraction of the ITER staff be seconded by the partners to the project for periods appropriate to their
tasks, and then return to their parent institutions to optimise the circulation of know-how .

Assuming this main strategy is implemented, and ITER is built, the following further recommendations
apply :

_8._ As soon as a firm decision on the construction of ITER is taken, the JET programme should be
reassessed and any possible, further activities concentrated on key issues for ITER, within the
stringent financial constraints which will be imposed by ITER construction. Particular attention
should be devoted to the management of JET personnel, as the termination of the project
approaches.

9 Association programmes on both physics research and technological development, should be pursued
with the aim of contributing to ITER and optimizing the database for DEMO. Physics research to be
conducted on small and intermediate size devices in the Associations, should deal with plasma
physics and concept improvement. Long-term technology work should include the development and
characterisation of appropriate reactor-relevant materials; this will require the construction of an

171

intense 14MeV neutron source. The design and development of the DEMO blanket should continue,
taking full advantage of the opportunity to test blanket modules in ITER.

The Board supports the development of the Stellarator line, so that future strategic decisions on
DEMO can be founded on a sufficiently broad physics basis.

The Board recommends that no new initiative be launched on the line of Reversed Field Pinch

devices as long as no new favourable data are available.

New "clustering" initiatives between the Associations around specific themes, similar to those already
successfully introduced, should be promoted.

_10._ The Board noted the substantial strengthening over recent years of the interaction between the Fusion
Programme and industry, centered mostly on ITER activities. Industry will have to play the primary
role in the final development of fusion: it is therefore recommended to increase progressively its
involvement, particularly in the System Engineering area.

In addition, the following recommendations are made regarding other aspects of the Programme's
activities:

_11._ The Board was favourably impressed by the work performed on the assessment of safety and
environmental aspects of fusion power and strongly encourages a renewed effort in this field.

_12._ A substantial effort should be devoted to socio-economic research on fusion. It needs to be multi
disciplinary and deal with issues such as: cost elements of fusion power, public awareness,
democratic governance of complex systems, and value change.

_13._ The watching brief on Inertial Confinement Fusion should be maintained, and in this frame a coordination of the civilian national efforts in Europe is recommended.

_14._ The high rate of human mobility achieved by the "Mobility Scheme" is essential to the cohesion of
the programme and should be maintained. The domain of application should be increasingly open to
industry.

75. The decentralised management of the Programme is efficient, flexible and well accepted by the
scientific, technological and industrial circles. With appropriate adaptations, it seems suitable for the
difficult tasks ahead. The ITER project will, of course, call for a specific, central management of
the European contributions when the project reaches the construction phase.

And finally, external events may invalidate the basic assumptions of this Report :

_16._ A breakdown of the ITER framework for any reason would result in the inability to go forward with
the planning and construction of ITER on the lines broadly outlined in this report. In this event the
Board recommends that there should be a complete reassessment of the European Fusion Programme
in the light of this new situation.

172

**II.** **COMMISSION'S** **RESPONSE**

1. At its meeting on 26 November 1996, the Consultative Committee for the Fusion Programme
(CCFP) has unanimously adopted the preliminary [1] views, herewith attached, on the Board
Findings and Recommendations. The Commission, which is a formal Member of the CCFP,
adheres fully to these views.

2. Regarding the Board recommendations on ITER, it could be added that the recent expression of
interest in hosting ITER from a EU Member State confirms, as pointed out by Mrs Cresson at the
Research Council on 7 October 1996, the will of Europe to play a significant role in this
enterprise. But the extent of this role cannot be defined before the financial involvement is fully
clarified. Quadripartite explorations have just started among the ITER partners which should
allow, among others, to draw up, in a year or so, realistic assumptions on the sharing of the
financial burdens between the potential host and the other partners.

**Preliminary views of the CCFP on the findings and recommendations of the 1996 Fusion evaluation**
**board**

- The CCFP welcomes the recognition by the Board of the success of R&D work performed and of the
validity of the strategy followed by the European Fusion Programme. It accepts the need for enhancing
industrial involvement, pursuing actively safety and environmental studies, and launching studies on
socio-economic aspects.

- The CCFP understands that arguments in support of the Findings and Recommendations will be found in
the full report.

- The CCFP wishes to highlight the following points from the Findings and Recommendations of the Board:

Fusion is mature for ITER, and the whole momentum of the Programme would be lost if ITER were
not built in the near future.

Today, Europe has the leading role in fusion research. Now that the construction of the first fusion
reactor approaches, with an increasing involvement of industry, it would be difficult to understand
that Europe abandons its leading role: ITER should be built in Europe.

Fusion has an important role to play in meeting the growing energy needs of the coming generations.
The CCFP agrees with the Board in pointing out that fusion has to be seen in the context of a much
wider energy research effort.

Following the principle of subsidiarity, not all energy options have to be developed at Community
level, but because of its long-term and wide scope, fusion necessarily does.

- The CCFP fully realizes that important additional work will have to be performed as a consequence of the
Board report:

In preparation of the ITER site decision, a thorough description of the long-term impact on both the
host and the non-host programmes, respectively, has to be produced.

The future programme will have to be re-directed and re-focused after the decision on ITER siting

and construction.

' The "Findings and Recommendations" of the 1996 Fusion evaluation Board are based on an in-depth analysis, performed by the
Board, which will be included in the full report yet to be finalised.

173

- The CCFP agrees with the Board that a negative decision on ITER would have such a large impact, that a
complete re-assessment of the European Fusion Programme would become necessary.

- The CCFP considers the Findings and Recommendations of the Board report to be an important input for
the preparation of the Fifth Framework Programme. When the full report of the Board will be available,
the CCFP will provide its Opinion.

174

**/18.**

**m .** **EVALUATION PANEL**

**Prof.** **Sergio BARABASCHI**

Former Under-Secretary of State for Research and Technology, Italy, Chairman of the European
Industrial Research Management Association (EIRMA)

**Dr** **Claus** **BERKE**

Former Executive Director at Siemens AG and former President of the European Nuclear Forum

**Dr Feliciano FUSTER JAUME**
Chairman and Chief Executive Officer of Empresa Nacional de Electricidad (ENDESA) and its Group of
companies

**Sir John HILL**
Fellow of the Royal Society, former Chairman, United Kingdom Atomic Energy Authority

**Prof.** **Lars INGELSTAM**

Professor of Technology and Social Change, Linkôping University

**Prof.** **Francis** **TROYON**

Professor of Physics, Ecole Polytechnique Fédérale de Lausanne, Chairman of the JET Council

**Prof.** **Harry van der LAAN**
Professor of Astronomy, Univesity of Utrecht, former Director-General, European Southern Observatory

**Prof.** **Jean-Paul WATTEAU**

Former Rector-Chancellor of Académie de Grenoble

175

19.

##### **1996 FIVE-YEAR ASSESSMENT** **JOINT RESEARCH CENTER**

OVERALL EVALUATION AND SYNTHESIS OF

VISITING GROUPS REPORTS

**by**

**Professor Juan M. Rojo**
**Universidad Complutense de Madrid**

**SPAIN**

177

**19.**

**L** **EXECUTIVE** **SUMMARY**

**I** have the clear impression _that further_ progress has taken place in the JRC since the encouraging analysis
of Sir Hermann Bondi three years ago. Both the management of the Centre and the Board of Governors
guidance deserve much credit for this steady evolution. It is also worth remarking that _most of the_
_recommendations_ made in the previous evaluation have been actually implemented. I am confident that
the recommendations that are included in the excellent reports elaborated by the present Visiting Groups
will also be of great benefit to the Institutes.

Under the guidelines of the Council, the JRC has obtained a much clearer _definition of goals_ and has
become much more _open and accessible_ to the outside, with increasing activities of support to the
Commission services and Union policies and _widespread networking with research organisations,_
_authorities and industries._ In particular, the Centre has reacted quickly to the new mandate of
_competitiveness_ on all fronts: it has successfully engaged in shared cost actions of the Framework
Programme of the Union and is becoming progressively more involved in joint ventures with industries.
With the refreshment of the management team, new and intensive drives towards project control, quality
management and marketing are being introduced _to increase the visibility_ of the scientific and technical
activities and to render the Centre's operation more transparent and understandable. Moreover, a change
of culture is discernible in which much of the staff now seems convinced of the benefits accruing from the
_more competitive_ stance of the JRC. Continuous support of this succesful policy is imperative and,
following the advice of some of the Visiting Groups, I recommend fostering it even further by setting up
_on a progressive basis_ a _user advisory panel for each Institute,_ that would promote relevance in project
missions and channel the needs of the users and their views on the operation of the Institutes, particularly
in relation to transfer.

In order to accomplish their mandated activities of institutional and competitive research the JRC has to be
an exceptionally qualified partner and it is gratifying to note that the JRC has succeeded in maintaining a
number of core areas of scientific excellence and is struggling to establish new ones. To nurture the search
for quality, I propose the creation of a permanent _scientific advisory panel_ reporting to the Board of
Governors. This panel would assess the quality of the ongoing research, with particular regard to
identifying high quality units in the Institutes, and would also advise on exploratory research. They would
also provide valuable inputs to the Visiting Groups.

An adversary of excellence is uncontrolled diversification. _Focusing_ is imperative as it seems that the JRC
has much too broad a spectrum of subjects. It is surely much better to have a restricted number of areas
in which the JRC is recognised as first-class than to have at one's disposal a kind of low-profile generalpurpose partner. After all, one must consider JRC activities with reference to subsidiarity and there is no
point in duplicating national facilities if there is no definite advantage. Much in the same line, the JRC has
to make every effort to avoid routine work, _even if it is requested by third_ _parties._

In order to maintain the positive evolution of the JRC, special attention should be given to the issues of
_staff management._ In relation to _recruitment,_ I have the most serious reservations for the future. As noted
in previous evaluations one has to remark yet again on the difficulties in trying to use a system apparently
designed to recruit government officials to obtain high grade scientists. It is far too rigid, and much too
slow to meet the demands of 1996, let alone those of the coming century. Some slight improvement has
been noted but a substantial overhaul is needed and I suggest the urgent setting up of a specific panel to
deal with the subject including possible changes of current regulations. I have great hopes that the recent
transformation of the JRC into an independent Directorate General might help to implement the specific
requirements of a research centre in the EC structure. _Mobility_ is another key issue that requires
revitalisation. The continuous flux of ideas to the Institutes should be encouraged by stimulating the
temporary attachment of trainee scientists from other institutions and the regular presence of research
students, particularly at the post-doctoral level.

179

19.

A final word about _energy._ I fully support the concerns of my distinguished predecessor regarding the
decline in nuclear relevant research. Independently of personal or political attitudes towards nuclear
energy in Member States, nuclear safety, waste treatment, non-proliferation and other issues, are still
with us. They are no respecters of national frontiers and deserve our attention at European level. It is
worth underlining that the JRC has first class expertise in some areas of this field.

In summary, I believe that the JRC is now on the right track and that present policies are workable and
satisfactory and, being supported by efficient management, have already transformed the Centre into a
useful partner of institutions and industry in Europe. In order to maintain this satisfactory trend a number
of steps, for which recommendations are given, should be taken, at least in relation to staff management,
support of quality, and focusing of research subjects.

**1.** **Introduction**

Research, technological development and demonstration (RTD) is one of the key policies of the present
Treaty of the Union, being third in rank in terms of economic volume. The reason for this high profile is
the conviction that those basic objectives of the European Union (EU), _industrial competitiveness and_
_improvement of the quality of life,_ can only be built on the solid foundations of science and technology.
Whereas the commitment of the EU to RTD has been permanent and increasing, the emphasis on the
different areas, or even in the type of activities required, has been continuously changing. The role of the
Joint Research Centre (JRC) has also undergone a number of realignments.

Following the dwindling interest in fission research in the past, specialised nuclear establishments in
Member States had to face arduous reconversions. The JRC was no exception and, in fact, some of its
present specialisations, like environment or safety, coincide with the goals of some of the reconverted
national nuclear centres. On these grounds it is not unexpected that in the past the JRC strategy was often
defensive, nor is it surprising that in the last few years, there have been a number of definitions from the
European decision-making bodies in relation to the goals to be attained by the Centre and to the various
procedures of evaluation and assessment of its performance. The last of them, in 1994, is the so-called
'modified role' of the JRC as agreed by the Council, Parliament and Commission. Among other issues this
new role emphasised the need for the JRC to be more involved in the denominated 'competitive actions',
often in partnership with enterprises.

Although the present assessment is supposed to cover the period of five years 1992-1996 (with full
recognition of the previous evaluation for the 1992-1994 period, the Bondi report), the introduction at
mid-term of the modified role of the JRC referred to above suggests that stress should be laid on the
performance of the Centre in terms of this _modified role,_ i.e. during the last two years. This has been well
understood by the Visiting^ Groups to the individual JRC institutes which, in their reports, have given
special attention to the implementation of the new more _competitive_ activities. These reports underline the
positive evolution of the JRC performance over the last two years, confirming the earlier expectations of
the Bondi assessment.

The significance of the individual reports of the Visiting Groups can hardly be over-emphasised as they
have been conducted in the most competent and professional way. Apart from a limited input derived
from personal meetings with Institute Directors and staff, most of the present assessment is based on those
reports. The scheme of my own report is as follows: In Section 2, I underline key points from the
individual reports, trying to look for unifying issues. I then proceed to a more general discussion of the
achievement of the JRC in the last five years (Section 3), keeping in mind the objectives defined in the
corresponding decisions of the European bodies. In Section 4, on the basis of the positive evolution of the
Centre, I look into the future in search of those R&D problems which the JRC can actually undertake
more efficiently than national laboratories. Finally, Section 5 deals with conclusions and
recommendations, the latter being general in scope. They are to be considered as complementary to the
specific ones suggested for the individual Institutes by the Visiting Groups.

180

**19.**

**2.** **Analytical remarks on the Individual Institutes**

The full reports of the individual Visiting Groups are published separately. They encompass an evaluation
of the activities of the Institutes, in the light of the objectives defined by the Council decisions, and
specific recommendations for the future. In the present Section 2, I attempt to select from those reports
some guidelines that I find most illuminating or representative of current issues.

**Institute for Reference Materials and Measurements** **(IRMM):**

- The Institute appears to occupy an important place in the EU structure of the sector. Indeed, it seems
that if the Institute did not exist, a centre of this type would have to be created.

- The Institute seems to have attained a satisfactory degree of competence and respectability, particularly
in nuclear reference and measurements. It also embodies special experimental facilities.

- Accountancy and management procedures require improvement and adaptation to the Institute's
particular role.

- Original research work must not be neglected. In this respect 10% of the resources seems a judicious
figure for exploratory research.

**Institute for Transuranium Elements** **(ITE)** :

- In my view, this is the most qualified Institute in the JRC. Every effort should be made to maintain this
status, particularly in recruitment issues in the face of coming retirements.

- The Institute has also been successful in incorporating competitive activities into their former, mostly
research oriented, ones.

- Avoid remaining in areas which become routine: rather transfer work to other establishments when
routine approaches. One should make sure that room for stimulation of creativity exists in all areas (i.e. in
safeguards analysis).

- In the two directions of exploratory research some novel ideas have originated recently in other
European laboratories (i.e. the 'energy amplifier and waste burner'). ITE should evaluate them, report to
the Commission and, if convenient, network.

**Institute for Advanced Materials** **(IAM)** :

 - Very much restructured, the Institute has gained recognition in the field of high temperature materials,
although there is still room for improvement in other fields. Diversification into already well-covered
fields should be avoided.

 - The future of the tritium handling laboratory (ETHEL) should be examined in view of the evolution of
the ITER and related fusion programmes. The relevance of this facility, in relation to existing ones in
Member States, should be clarified.

 - Concern is expressed in a decrease in researchers being exchanged with external well-known
institutions, compared to an earlier situation. This seems to me a critical issue.

181

19.

- Transfer to the outside seems to be improving although research at the Institute should better remain
generic and near, but upstream, of industrial research. The Visiting Group considers that shared cost
actions should be favoured and encouraged.

**Institute for** Systems, **Informatics and Safety (ISIS)** :

- This Institute has been formed from the merging of two former ones, a new Director has been appointed
and is, naturally, still defining goals and procedures. The management showed good perception as well as
determination and should reap future rewards.

- In recent times, the Institute has made a great effort in working together with external research and
industrial bodies and this competitive attitude has to be stimulated and supported. The proposed exchanges
of personnel with Directorates General in Brussels appear as a good practice for reinforcing a users
approach with these units.

- Quality seems to be generally satisfactory. Perhaps the very wide spectrum of interests of the Institute
could be modulated by actual capabilities at the highest level. In some areas, improving scientific
leadership seems desirable.

- The Institute includes some unique facilities whose maintenance at the highest level of performance is
mandatory. Enhanced use of these facilities by outside users is recognised.

**Environment Institute** **(EI)** :

- The Institute seems to be on the right track to the fulfilment of both institutional and competitive
activities with a fair level of quality. The willingness of the management to convert it into 'the leading
institute in Europe' with 'a high degree of visibility' has to be made compatible with increasing activities
at national level in these areas and certainly requires careful focusing of the actions.

- Along with a proper management control the management seems to be successful in stimulating the
generation of new ideas, without which institutional and competitive activities could not be carried out
efficiently.

- Institutional activities are crucial in this domain. For example, the EI has produced amazing results
from the comparison of pollution measurements in blind tests performed by different national reference
laboratories with a scatter of more than 50%. It is hard to overemphasise the incidence of this work on
implementation of environmental policies.

- I strongly support the Visiting Group's suggestion of developing to capacity the activities in water
research and management of water resources. An independent and neutral reference, along with the
required scientific quality, is mandatory in this field.

**Space Applications Institute** **(SAI):**

 - Institute with unique experience on télédétection and earth observation. It would benefit from stricter
control of scientific quality (stimulate publications in prestigious journals) and performance in competitive
projects (better definition of specific goals and milestones).

 - As remote sensing is generic by nature and aids in resolving so many different problems, one has to
restrain the tendency of the Institute to become specialised in too wide a range of subjects. It would be
better to restrict activities to the Institute's core competences (remote sensing and application of networks).

 - Interdisciplinarity requires improved co-operation with other Institutes, particularly the Environment

Institute.

182

**19.**

**Institute for Prospective Technological Studies (IPTS):**

- Practically new-born, the Institute is starting to establish a sound reputation with its customers and set up
infrastructure to meet their future demands. The location needs communications and world-wide-web to

be developed to full strength. The Institute has still to demonstrate high quality and gain full recognition.

- Previous recommendations on recruitment for Seville and launching of the European Science and
Technology Observatory (ESTO) have been accomplished.

- The general recommendations on 'focusing' are still more meaningful for IPTS. Work should not be
encouraged in areas where European industries and laboratories have a long established tradition.

- The Institute is taking advantage of its geographical position for developing leadership in Mediterranean
and North African problems. I firmly believe that this approach should be reinforced, perhaps, as the
Visiting Group indicated, by developing a programme outside the TEC (Technology, Employment,
Competitiveness).

**3.** **General assessment of the JRC achievement**

In his report for the period 1992-1994, Sir Hermann Bondi stated that 'the JRC is well on the way to
becoming a recognised centre of excellence in many of its areas of activity'. It is a pleasure to be able to
confirm steady progress towards this desirable goal two years later. By taking together the five year
period, from 1992 till the present, one can safely say that the JRC has successfully found a specific
position in the European RTD system and that it is performing its role with increasing capabilities.

Under the guidelines of the Council the JRC has attained a much clearer _definition of goals_ and has
become much more _open and accessible_ to the outside, with increasing activities of support to the
Commission services and Union policies, and in widespread networking with research organisations,
authorities and industries. With the refreshment of the management team, new and intensive drives
towards project control, quality management and marketing are being introduced to _increase the visibility_
of the scientific and technical activities and to render the Centre's operation more transparent and
understandable Moreover, a change of culture is discernible in which much of the staff now seem
convinced of the benefits accruing from the _more competitive_ stance of the JRC. It is perhaps too early to
assess the impact of new distribution of tasks among the Institutes, as some merging and reorientation has
recently taken place, but it is clear that further strengthenening the coordination among the Institutes is
desirable.

It is to be remarked that business-like management can never be at odds with scientific and technological
_quality._ One cannot speak of transfer if there are neither significant ideas or know-how to be transferred.
An unreasonable fixation on demonstrating 'usefulness' should not obscure the fact that in order to gain
competitiveness, recognition and selected support of high quality units in the Institutes should be assigned
a high priority. In fact, the JRC has succeeded in maintaining a number of core areas of scientific
excellence and is struggling to establish new ones, although much thought has to be given to recruitment
policy if one wants to maintain that promising trend. Also, the JRC seems to have been evolving into _less,_
_bureaucratic_ and more transparent managerial procedures. With regard to both items, it is satisfactory
that the JRC has recently become an independent Directorate General as it is hard to use a system
apparently designed to manage administrative offices to run an RTD centre.

Whereas the present situation of the JRC can be regarded with confidence, there are a number of issues
that are still a matter of concern. I refer to the most important ones perceived by the Visiting Groups, and
in full agreement with my own experience:

1. _Personnel._ A matter of common concern as this is certainly the _key_ point for the future of the JRC.
One should make sure that the massive retirement that is expected in the coming years is offset by hiring
younger people of the highest quality. Recruitment procedures are still far from efficient. The whole

183

19.

process of recruitment should be, tuned so that 3 years contracts could be used as a test of JttD capabilities
for future candidates to temporary (semi-permanent) positions. I believe that only an overall evaluation of
the problem can lead to substantial improvement and I specifically advise setting up a special panel for this
purpose. As is often stated that much needed reforms are contrary to EU rules, maybe changing some of
these rules for RTD centres could be one of the recommendations of that panel.

2. _Exchange_ _of_ _researchers._ An action much praised by some of the Institutes in the past, particularly in
strengthening capabilities and training young researchers in emerging areas. It also facilitates networking.
However, it seems to be receding and correction of this trend should be undertaken.

3. _Focusing._ There seems to be much too broad a spectrum of subjects. It is surely much better to have
a restricted number of areas in which the JRC is recognised as first class than to have at one's disposal a
kind of low-profile, general-purpose partner.

4. _Management and_ _Accountancy._ Some advance since the last report is recognised (i.e. rapid response to
manpower needs in third party projects). However, improvements are still widely requested. An
appraisal of general accountancy procedures is suggested.

5. _Co-ordination._ Further advances since the last report are recognised, but stronger co-ordination among
the Institutes is required. The Directorate of Programmes seems to be working at a very reduced staff
level and requires examination.

_6. Nuclear energy._ Apart from its historical origins, the JRC is the depository of much European
knowledge in this subject, perhaps the only one in which it is recognised as the true European leader in
some areas. In no way should this capital asset be wasted.

**4.** **A look into the future**

The JRC has received the mandate of performing three types of activity: institutional, competitive inside
the Framework Programme (FP) and competitive outside the FP. Positive response in the period under
examination to this challenge suggests that the time may be ripe for a further step, which would be that of
progressively concentrating JRC actions on those in which its _European added value_ (EAV) is fully
exploited. In other terms it would mean proceeding from the question: Is there anything useful that we
can do with the infrastructure and personnel in the JRC? to the much more engaging one: Are there
R&D areas or activities in which the existence of a European-owned research laboratory is desirable or
even necessary? Indeed, the overall RTD policies of the European Union (EU) are built on the principle
of subsidiarity. As is well known, the EU Member States have a number of excellent laboratories, both at
their universities or national institutes and at their industries. One may wonder whether there is in
principle scope for an institute at European level, in which _targeted research_ is performed. If the functions
of such a centre can be recognised, one can further analyse if the JRC is presently capable of fulfilling
these requirements.

On the basis of the Council Conclusions of 26 April, 1994 [1] one could envisage three possible domains in
which the JRC would have full EAV:

The _first_ domain would be that of specific and difficult problems that required, for their solution, _critical_
_mass_ of scientific and technical _excellence._ Excellence would be promoted by drawing from the pool of
bright people in the different Member States and integrating that talent in first-class units of critical mass,
that would furthermore pervade through the European RTD system by means of suitable networking and
partnership. The _second_ domain would be that of developing, maintaining and managing _large_
_experimental_ _facilities,_ which could not be run independently by Member States. In the past a number of
large experimental facilities have been set up, or upgraded, in Europe on the basis of transnational cooperation among different countries. The _third_ domain would be that of carrying out research and

OJNo. C126, 7.5.1994, p.l

184

19.

development actions in which _impartiality_ and _neutrality_ were mandatory. Needless to say, these virtues
would have no practical use if they were not accompanied by scientific quality and recognition.

Let us now turn to the crucial question, that is if one can realistically consider that the JRC can be
competent in all these three domains at the present time:

- _First,_ its possible role as hosting _units of excellence_ is analysed. Needless to say, I refer to both
_scientific_ and _technical_ excellence. Gaining excellence is not at all an easy task: if an institute has
little prestige, it will hardly attract the brightest people, which in turn will result in further
stagnation. One has to recognise that the JRC has rarely become a reference point in the past, but
the situation is steadily improving as discussed in Section 3. The present management has
succeeded in injecting morale and determination into the staff, which now seems to believe in its
own capabilities. Nevertheless, some bureaucratic remnants are still noticeable; for example, some
officers when describing their work appear to be seized by the _how_ but reluctant to emphasise the
_what._ Full support of emerging new and promising ideas should be prevalent in all the Institutes.

As support must always be selective, the management of the Institutes would much benefit from
accurate information concerning the external appraisal of the quality of the different groups. I
believe that the Board of Governors (and the Visiting Groups for their periodic assessment) would
very much benefit from the creation of a _scientific advisory panel_ that assessed the quality of the
ongoing research, with particular regard to identifying high quality units in the Institutes, and
advised also on exploratory research. One must not forget that only high quality units can actually
provide EAV to their competitive partnerships with European industries and institutions.

In the pursuit of excellence, there are a number of issues in which improvement is desirable.
Ensuring that the best young people apply and that selection is efficient is second to none among
the elements to maintaining high standards. My advice about personnel reforms is here crucial, as
is my recommendation about focusing. A further suggestion in relation to mobility also seems
relevant. It is always difficult to attract on a permanent basis first-class scientists or technologists
who are well established in other institutions. However, incorporating them into the JRC groups
for limited (but not too short) periods could result in immense benefit. Perhaps, previous practices
of the Institutes to incorporate external researchers could be reinforced by developing a special
programme which took advantage of sabbatical leave and included some convenient incentives for
attracting these highly rated researchers.

 - Let me now turn to the _second_ function, that of _managing_ _large_ _facilities_, which can become also
reference points for networking with other European institutions. It is worth remarking that none
of the recently developed large facilities in Europe has been undertaken under the JRC umbrella.
Even the first European tokamak (JET), although funded by the Framework Programme of the
EU, was conceived and developed outside the JRC. The positive evolution of the JRC,
continuously alluded across this report, might change the attitudes of the Member States in the
future and some thought should be given to alleviate some of the pending obstacles. The new
Articles 130 k, 1 and n of the Union Treaty might also help to implement this function.

 - Finally, I address the function of _impartiality_ and _neutrality._ It is often stated that Union policies
require support from the JRC but very often this support is thought of in terms of routine or
bureaucratic work. This is certainly wrong and creative and original work is mandatory for these
purposes. This is perhaps the role in which the JRC has made a most visible progress recently.
The support of this role should continue, but one must not forget that this support would become
meaningless if the quality of JRC research failed and, consequently, a subtle equilibrium between
the first and third domains is imperative.

Last, but not least, the development of a body of independent knowledge in scientific and
technological prospection is crucial for complementing the inputs from the different Member States
in order to formulate the basis of future Framework Programmes. Previous Framework

185

19.

Programmes have suffered from the lack of a neutral and recognised scientific and technological
prospection and some of their programmes have become a mere list of Member States' interests,
often without much EAV. The JRC has still to gain prestige and full recognition in this task but
the rewards can be vast.

**5. Conclusions and Recommendations**

**I** have the clear impression that considerable _further_ progress has taken place in the JRC since the
encouraging analysis of Sir Hermann Bondi. Both the management of the Centre and the Board of
Governors guidance deserves much credit for this steady evolution. In particular, it is worth remarking
that _most of the recommendations_ made in the previous evaluation have been actually implemented.

The mandate to enter the competitive arena, and the general positive answer to this challenge in the
different Institutes, has had a number of favourable consequences:

0 Providing a check of quality (competing with national teams, having to attract potential users etc.)

0 Opening the JRC groups to the outside, i.e. networking, etc.
0 Improving management procedures, like quality management, project management and active
marketing.

The first recommendation is, consequently, that of _maintaining and supporting the present policies._ One
has to realise, however, that even for consolidating the present advances, a number of improvements have
to be made, particularly in the area of personnel management. Also, looking into the future, one has to
aim at establishing the JRC as an institution with full EAV in the European RTD community and this will
require further adjustments. In the following, I summarize a number of recommendations that derive from
the analysis of the previous sections:

a) Recommendations **for specific action.**

    - Set up _urgently_ a special panel of experts, including specialists in European legislation, to
advise on the subject of _personnel_ status and recruitment procedures, _including_ possible
changes of current regulations.

    - Explore the possibilities of developing a special programme for the incorporation of first-class
scientists into JRC laboratories during their sabbatical leave, that would reinforce previous
individual action by the Institutes. Ensure a continuous incorporation of research students,
particularly at post-doctorate level to the Institutes.

    - Set up an external _scientific advisory panel,_ reporting to the Board of Governors, that would
assess the quality of the ongoing research, with particular regard to identifying high quality
units in the Institutes, and advise also on exploratory research. The panel would also help with
the selection of new scientific staff and exchanges.

    - Set up _user advisory panels_ at the Institutes in which the convenience of their existence has
been manifested (for example, ISIS) and, on the basis of ongoing experience, extend them to all
the Institutes.

b) Recommendations **for general policy**

    - Avoid excessive proliferation of subjects in the Institutes. Instead, try focusing on activities
leading to attainment of first-rate status in a limited number of subjects.

    - Emphasise, in the current accounts of the JRC activities, _concrete_ results obtained in **JRC**
Institutes rather than organisational issues. Periodically publish and disseminate news about
those achievements. If possible select those cases in which the JRC has been given _significant_
recognition (as opposed to 'listing customers').

186

    - Stimulate an ambience in which original, promising ideas can arise and support them in order
to explore their possibilities Encourage publication in prestigious journals in order to gain
recognition. Ensure that sufficient effort is devoted to exploratory research.

    - Avoid routine work _even if_ _it_ _is_ _sought_ _by_ _third_ _parties._ Try to step out of certain lines, and
leave their continuation to other centres once the EAV of the action has vanished. Nevertheless,
be careful in this respect in those areas in which the JRC has _unique_ facilities.

    - Undertake the necessary steps to support the JRC as a candidate for future European large
facilities. Explore the possibilities of using Articles 130 k, 1 and n as a 'driving force'.

**A note on the evaluation process itself**

In general, the evaluation process itself seems to fulfil its original purpose. In fact, previous evaluations
have actually contributed to marked improvements in different aspects of JRC management. There are,
nevertheless, aspects that can be ameliorated:

     - The Visiting Groups' task could be supported by an external evaluation of the quality of the
output of the Institutes and by a unified view of the users appreciation of transfer, competitive
partnership and related matters. The proposed panels could well fill this gap.

    - The first point of the terms of reference of the Visiting Groups states that the evaluation refers
to _all_ the objectives of the Council Decisions of 15 December, 1994 . However, the Visiting
Groups' reports appear rather unbalanced in relation to the different objectives; for example, the
sixth objective of the JRC "contributing to the reduction of scientific and technological disparities
between Member States" does not seem to have played an easily recognisable role in the
evaluation.

**Acknowledgements**

I acknowledge very useful discussions with the Board of Governors and members of the Management of
the Centre and of the Visiting Groups. The exchange of views on the report with Sir Hermann Bondi has
been of immense benefit to me.

OJ No. L361, 31.12.94, p.l 14 and p.132

187

**19.**

**11^** **MEMBERS OF THE VISITING GROUPS**

**A.** **Institute for Reference Materials and Measurements (IRMM) - Geel**

**Dr. Maria Grazia DEL MONTE** (Chairwoman)
Director

National Centre for Reference Materials

ITALY - ROME

**Mr. Peter J. JENKS**
Product Manager
Biological, Elemental and
Pharmaceutical Reference Materials

Promochem Gmbh

GERMANY - WESEL

**Mr. Robert KAARLS**

Director

Nederlands Meetinstituut (NMi)

THE NETHERLANDS - DELFT

**B.** **Institute for Transuranium Elements (ITE) - Karlsruhe**

**Dr. Niels** HANSEN(Chairman)

Head

Materials Department
Riso National Laboratory
DENMARK - ROSKILDE

**Mr. Richard HOOPER**

Director

IAEA

AUSTRIA - VIENNA

**Dr.** Klaus **JANBERG**

Geschaftsfuhrer

GNS

Gesellschaft fur Nuklear-Service GmbH

GERMANY - ESSEN

**Mr. Jean van** **VLIET**

Manager
MOX plant
Belgonucléaire
BELGIUM - DESSEL

**C.** **Institute for Advanced Materials** **(IAM)** **- Petten and Ispra**

**Prof.** **Claude BATHIAS** (Chairman)
Conservatoire National des Arts

et Métiers (CNAM)
FRANCE - PARIS

188

**19.**

**Prof.** **Pier Giulio** **AVANZINI**

Direzione Area della Ricerca
Consiglio Nazionale Ricerche
ITALY - GENOA

**Mr. Staffan SÔDERBERG**

Director

AB Sandvik Coromant

SWEDEN - STOCKHOLM

**Prof.** **Dr. T. VARGA**
Leiter der Techn. Versuchsund Forschungsanstalt
TU Wien

AUSTRIA - WIEN

**D.** **Institute for Systems, Informatics and Safety (ISIS) - Ispra**

**Mr. Peter BENTON** (Chairman)
Director General
British Institute of Management
UNITED KINGDOM - LONDON

**Prof.** **Jane GRIMSON**
Dean of Engineering and Systems Sciences
Department of Computer Science
Trinity College
IRELAND - DUBLIN

**Mr. Theodossios P. TASSIOS**
National Technical University of Athens
GREECE - ATHENS

**Prof.** **Dr. Dennis TSICHRITZIS**

Chairman of the Executive Board
GMD-Forschungszentrum
Informationstechnik GmbH

GERNANY - SANKT AUGUSTIN

**Dr. F. YNDURAIN**

Director General

CIEMAT

SPAIN - MADRID

E. **Environment Institute** **(EI)** **- Ispra**

**Prof.** **Carlos A. BORREGO** (Chairman)
Department of Environment and Planning
University of Aveiro
PORTUGAL - AVEIRO

**Prof.** **A.R. HARDY**

Research Director
Central Science Laboratory
UNITED KINGDOM - SLOUGH BERKS

189

**19.**

**Prof.** **Dr. A. KETTRUP**

Director

GSF Neuherberg
Institute of Ecological Chemistry
GERMANY - NEUHERBERG

**F.** **Space Applications Institute (SAI) - Ispra**

**Prof.** **Seppo URPO** (Chairman)
Director

Metsâhovi Radio Research Station

FINLAND - KYLMALA

**Prof.** **Dr. Herbert FISCHER**

Head

Institut fur Météorologie und Klimaforschung
Forschungszentrum Karlsruhe
GERMANY - KARLSRUHE

**Mr. Roy GIBSON**
Former Director General of ESA

FRANCE - MONTPELLIER

**G.** **Institute for Prospective Technological Studies (IPTS) - Seville**

**Dr. Robert C. WHELAN** (Chairman)

Former Chief Executive of CEST

UNITED KINGDOM - CAMBRIDGE

**Dr. Josef FRÔHLICH**
Head, Division of Systems Research
Austrian Research Centre Seibersdorf

Technology-Economy-Environment
AUSTRIA - SEIBERSDORF

**Prof.** **Dr. Herbert PASCHEN**

Leiter des Instituts fur Angewandte Systemanalyse
und Technikfolgenabschàtzung (ITAS)
GERMANY - KARLSRUHE

**Prof.** **Walter ZEGVELD**

Chairman

Rathenau Institute

THE NETHERLANDS - THE HAGUE

190

Annex

191

**ANNEX** **1**

**TERMS OF REFERENCE**

**FOR THE 5-YEAR ASSESSMENT OF THE**

**SPECIFIC PROGRAMMES**

**1.** Assessing relevance of programme objectives :

Are the initial objectives still valid against evolving S&T, industrial and socioeconomic conditions ?
Have the selection criteria as outlined in the Framework Programme Decisions
been adhered to ?

2. Assessing efficiency :

Has the programme sought to achieve/achieved its objectives in a cost effective
manner ?
How efficient has the overall management of the programme been ?
Have objectives been adopted against which performance can be evaluated ?
Is improvement to the external monitoring process/procedure necessary ?

3. Assessing effectiveness :

Have the initial objectives of the programme been achieved, or, for the longer
term strategy and objectives, is progress towards achieving them sufficient ?
Have the results stemming from the programme been disseminated/exploited ?
Have the overall objectives of the Framework Programme been achieved,
particularly :

        - What have been the contributions to strengthening the scientific and
technological bases ?

        - What has been the interaction with and/or what have been the contributions

to relevant Community policies such as cohesion, competitiveness, agriculture,
employment, cooperation for development, economic cooperation, fisheries, etc. ?

        - Is Community RTD activity sufficiently coordinated with that of the Member
States ?
How is European "added value "/cost benefit/subsidiarily best demonstrated ?
Have the results stemming from the programme been disseminated/exploited ?

4. Examples of major achievements :

Outstanding S&T results and innovations.
Major achievements as regard the contributions to other community policies.

5. Lessons learned from programme implementation :

What lessons have been learned from programme implementation ?
Success stories of collaboration.

Failure stories of collaboration.

6. Recommendations for the next programme.

The five-year assessment will pay particular attention to the coherence between
Community andnational S&T policies with a view to enhancing their mutual consistency.

192

**ANNEX 2**

**TERMS OF REFERENCE**

**FOR THE** **VISITING** **GROUPS FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S INSTITUTES**

1. To evaluate progress in performing work in accordance with the objectives set out in Council Decisions
94/918/EC and 94/919/Euratom, documents addressed to the Board of Governors, the JRC's Annual
Work schedules, budget etc., taking into account the results of the last JRC evaluation performed in
1993-94.

2. To ensure that the Institute has effective mechanisms to set its local scientific objectives, monitor
progress towards those objectives and evaluate outputs.

3. To review the various research activities to assess:

a) the relevance of the work with respect to Community needs, including the relevance of work
performed for external customers, when applicable. In particular whether the objectives
initially set for the Institute are still valid against evolving S&T, industrial and socio-economic
conditions;

b) the scientific quality and productivity of the Institute;

c) whether full advantage is being taken of scientific opportunities in the light of funding and other
constraints.

4. To assess the effectiveness of the scientific management leadership within the Institute.

5. To advice on whether the Institute has been making efficient, effective and economical use of resources
in carrying out its programmes and management functions. Resources include manpower, money,
services, facilities, data and equipment.

6. To review the extent and effectiveness of the Institute's external scientific links, including cooperation
with research organisations, the higher education sector, industry and government departments in the
Member States.

7. To examine the relationship of the Institute's work in general, to the mission of the Joint Research
Centre and to its forward strategic planning as well as to the overall objectives of the Framework
Programme. Therefore assessing :

    - the contribution towards the improvement of the competitiveness of European industry and the
economic impact (both direct and indirect) that has already been plus that which is expected to
be achieved;

    - The contributions to strengthening the scientific and technological bases;

    - The contributions to relevant Community policies such as environment, industry, agriculture,
transport, employment, cohesion, etc.

To assess the European added value of the Institute's activities on the basis of the subsidiarity
principle.

8. To make recommendations and report to the Board of Governors of the Joint Research Centre.

193

###### ISSN 0254-1475

### COM(97) 149 final

# **DOCUMENTS**

##### EN 12 15 16 05 Catalogue number : CB-CO-97-140-EN-C ISBN 92-78-18077-7

Office for Official Publications of the European Communities

L-2985 Luxembourg