Source: EURLEX
Language: en
Format: md

**COMMISSION** **OF THE EUROPEAN** **COMVfUNITIES**

```
                             COM(95) 6 0 final

                             Brussels, 08.03.1995

```

**Evaluation of the Joint Research Centre and its**
**Muitiannual Research Programme**
**1992-1994**

_**A**_

Opinion of the Board of Governors of the Joint Research Centre

on the Evaluation of the Centre and its 1992-1994

Muitiannual Research Programme

Introduction

Each of the Council Decisions 92/273/.EEC and 92/274/Euratom of 29th April
1992* establishing the 1992-1994 Muitiannual Research Programme for the JRC,
stipulates in Article 5 that:

" 1. Research carried out by the JRC will be evaluated by a group of independent
external experts set up by the Commission after consulting the Board of
Governors. An evaluation report on the subject shall be established at the
end of the programme.

2. The evaluation report referred to in paragraph 1, accompanied by the
opinion of the Board of Governors, shall be forwarded by the Commission to
the European Parliament, the Council and the Economic and Social
Committee."

At the same time, the Supplementary Programme for the Exploitation of the High
Flux Reactor, has also been taken into account with a similar evaluation
procedure being applied.

Following recommendations made by the Evaluation Panel under the
chairmanship of Sir Hermann Bondi, which evaluated the JRC's 1988-1991
programme, the Board of Governors, in consultation with the Director General of
the JRC and in agreement with th.e Commissioner responsible for science, research
and development, decided to set up a number of Visiting Groups of independent
senior scientists and science managers to examine the JRC on an institute-byinstitute basis. The reports arising from these surveys, together with a general
analysis by Sir Hermann Bondi which draws together the conclusions of all these
reports, are found annexed to this Opinion. These documents comprise the body
of evidence of the present state of the JRC and its Institutes as seen by outside
observers and, as well as fulfilling its statutory purpose, is an important base for
management purposes and a significant input to strategic planning of future JRC
scientific and technical activities. Equally important are judgements on research
quality and efficiency in JRC Institutes and suggestions on scientific and
organisational matters.

The Board wishes to express its thanks to the many eminent scientists and
managers who freely gave so much time to the visits and preparation of the
Reports, especially the Chairmen of the Visiting Groups. A special vote of thanks
is also due to Sir Hermann Bondi for his critical analysis and summing-up.

- Official Journal of the European Communities NT I 141 of 23 May 1992

**2-**
_**V**_

**Opinion**

**The Board of Governors has examined the Reports and shares Sir** **Hermann's**
**impressions of continuous improvement of the JRC since its reorganisation in**
**1988,** **particularly with respect to the opening up of the JRC to the outside world**
**and to its internal functioning. It is also satisfied with the success of the Visiting**
**Groups, both in their evaluations of Institutes and as a means of providing**
**Directors with helpful and informed advice on scientific and operational matters.**
**However, for future groups, a greater emphasis in their evaluations on the value**
**of results of research and their impact would be desirable.**

**The Board notes the expressed need for tighter project definition and for**
**improved recruitment procedures which has been the aim of the Board, who shall**
**strive for improvement in these fields.** **However, the Board has noted**
**considerable progress in the first field and less in the second field in recent years.**
**The Board believe there is a high awareness of the problem at JRC management**
**level and the Board shall continue to support the efforts in this direction.**

**The Board recognises and upholds the efforts by the Director General and his staff**
**to respond to recommendations made in previous evaluation reports and is happy**
**to see that these efforts are bearing fruit. Renewed efforts will be undertaken by**
**the Board to adapt the organisation to its new and more competitive place in the**
**scientific** **community** **and the Board counts on the universal acceptance in and the**
**support of the Commission along that line.**

**In this respect, it is clear that, there are certain limitations imposed by the**
**Commission's financial and staff regulations which preclude the implementation**
**of some of the recommendations, which are common practice in a free market.**
**The Board recognises these restraints but will continue to encourage and**
**recommend simplification and greater flexibility wherever** **possible** **and notes**
**with satisfaction that most of the simpler recommendations are already being put**
**into effect and that serious efforts are being applied to the more difficult JRC-**
**wide problems that have been raised. Experience will show if this is sufficient to**
**keep the JRC competitive and the Board will closely monitor the progress.**

**After seeing the results of this first experiment, the Board is ready to continue the**
**Visiting Group system for future evaluation and management advice and to press**
**for maximum implementation of forthcoming advice.**

**Cambridge, 2nd September, 1994**

**ANALYSIS OF THE REPORTS OF THE**

**VISITING GROUPS TO THE**

**JOINT** **RESEARCH** **CENTRE'S INSTITUTES**

_**(JRC MULTIANNUAL RESEARCH**_
_**PROGRAMME**_ _**1992-1994)**_

**Professor Sir Hermann Bondi**

**4**

**Analysis of the Reports of the Visiting Groups to the Joint Research**
**Centre's Institutes** _**(JRC**_ _**Muitiannual**_ _**Research**_ _**Programme 1992-1994)**_

**Executive Summary**

**1.** **The strongest impression one receives from studying the Panel Report of**
**1991,** **the nine Visiting** **Groups'** **reports of 1993-1994 and my report of June**
**1994 (together with my letter to the Chairman of the JRC's Board of**
**Governors) is on the.continuous positive development and evolution of the**
**external links of the JRC. Under the energetic leadership of the Director**
**General,** **advice has been accepted and acted upon with impressive speed.**
**This has been of great benefit to many individual scientists, to the JRC, to**
**the** **Commission** **and** **to** **the** **general** **European** **industrial-academic**
**atmosphere.**

**2.** **It is most gratifying that the institution of Visiting Groups has been so**
**successful. In each case, the JRC has established valuable new connections,**
**received constructive advice and made knowledgeable friends.** **It** **is also**
**remarkable how similar the Visiting Group reports are to each other.**
**Scientific standards have received nearly universal praise, institutional**
**support to the Commission is judged to be good and improving, but with**
**further development still attainable. The stimulus of having to gain work in**
**a competitive manner is regarded as very positive, but the** **necessary** **change**
**of attitudes and indeed culture is, for some Institutes, quite considerable**
**and needs sustained determination. The speedy growths in external links**
**through** **students, visiting** **scientists, networking,** **etc.** **is universally**
**applauded and further growth is recommended.**

**3.** **Internal management of the Institutes is widely praised, but more progress is**
**needed in some places through introducing tighter project definition and**
**control.** **In** **particular, a significant strengthening of the management of**
**tasks involving more than one Institute is seen as an urgent need.**

**4.** **Almost every Visiting Group is deeply worried about staff recruitment**
**procedures, and several would like to see some improvement in internal**
**staff management, entirely in line with my views as expressed in 1991 and**
**more recently.** **I** **cannot say too strongly that one cannot be optimistic about**
**the future quality of the JRC, especially after the retirement bulge of the**
**late nineties, unless there is a major revolution in recruitment methods,**
**yielding a substantial speeding-up and a much greater involvement of local**
**experts. As** **I** **have said previously, a posteriori judgment of choices made can**
**meet requirements of excellence and of Member States' wishes far more**
**effectively than the present cumbersome and wasteful procedures.**

**5.** **Finally,** **I** **must express my concern about pressures for a reduction in nuclear-**
**relevant** **research.** **I** **fully appreciate the political strength of anti-nuclear**
**views in several Member States. It must be realised that there are three**
**powerful reasons for at least maintaining nuclear research:**

**(i)** **even without a single new nuclear power station, Europe will have to**
**deal with nuclear waste for a long time to come;**
**(ii)** **concern about the safety of power stations in Eastern Europe will not**
**go away;**
**(iii)** **worries about nuclear weapon proliferation are growing all the time**
**and Europe should continue to be able to play its part in the required**
**scientific detective work.**

_**c**_ **- 2 -**

**Report**

**Introduction**

**1.** **In** **1991** **I** **chaired a panel set up by the Board of Governors of the Joint**
**Research Centre to evaluate the Centre and the work performed during the**
**1988-1991 Muitiannual Research Programme** **under** **the requirements of the**
**Council Decisions taken** **in** **1988 and establishing that Programme. In the**
**ensuing report, issued on 17th January 1992, the Panel proposed a change in**
**the mechanism to be used for evaluating the JRC in order that the rather**
**diverse activities of the various Institutes could be properly examined in**
**somewhat greater detail by groups of independent, acknowledged experts**
**working in the relevant fields, and to forge closer links with leading**
**European scientific organisations active in related domains.**

**Accordingly, during 1993 and 1994, nine expert Visiting** **Groups have**
**evaluated the eight JRC Institutes, an extra** **Group** **examining the reactor**
**HFR,** **and have drawn up reports of their findings for the Board of**
**Governors.**

**During the first half of 1994, the present author** **met** **the Board of Governors**
**and visited each of the Institutes and JRC headquarters in Brussels to advise**
**on the development of a strategy. This was in response to the Council**
**Decision to open the JRC to competitive participation in Shared Cost Actions**
**and to competition with external research** **organisations** **in providing**
**scientific** **services to other Directorates General of the Commission.**

**2.** **The present report sets out to present an overall snapshot of the JRC**
**through the eyes of Visiting Groups and revealed in their reports, with**
**comments based on the author's past and present perception of the** **JRC.**

**The scope and terms of reference of the Groups is found in Annex** **I** **of this**
**report which includes the requirements laid down in Council Decisions and**
**the Resolution on the JRC 1992-1994 Muitiannual Research Programme**
**regarding the evaluation of the work carried out. At the same time, Groups**
**were** **required** **to** **review** **progress** **in** **the** **implementation** **of**
**recommendations of the above -mentioned Evaluation Panel, to judge the**
**scientific quality, appropriateness, efficiency and impact of** **Institutes'**
**activities and to examine an Institute's strategy for the future.**

**General Points**

**3.** **The period since 1991 has been one of opening out, to the general benefit**
**of the value of the JRC to the scientific and industrial communities. The**
**Visiting Groups have given the Institutes much better links with the outside**
**world,** **the growth in numbers of postgraduates,** **postdoctorates** **and more**
**senior visitors has been substantial and of great value in many ways, and the**
**increase in external contracts is not only a necessity, but of great benefit to**
**the scientific vitality of the Centre.** **All this is entirely in line with the**
**recommendations of the 1991 Panel and most gratifying. This approbation**
**seems to be shared by all Visiting Groups, not only for the advantages it**
**brings to the JRC but also for the valuable training it provides. Nevertheless,**
**it is felt that the success of the action warrants the active pursuit of all means**
**of opening the JRC.** **Networking with peer organisations is strongly**

**-3**

**encouraged by all Groups, although the rather limited mobility of staff,**
**both within the JRC and with the outside world requires serious attention. It**
**has been suggested that promotion above a certain level should be**
**dependent on service for a number of years in more than one country.**
**Moreover,** **my** **own experience has been that in many cases, staff at about 45**
**or 50 years of age become more mobile as their children reach university age**
**or become independent.** **It** **could also be considered** **unreasonable** **to pay a**
**foreign** **service^** **I Iowa nee** **to staff remaining in one country for longer than**
**ten to fifteen years.**

**Particularly noted is the near absence of a modern career development**
**strategy with an attendant training plan applicable to all staff.**

**Another positive sign of the JRC having become a** **more** **open and valuable**
**Commission** **tool,** **is the way in which activities in support of Commission**
**sectorial policies have grown in recent years. In part, this trend can be**
**ascribed to a response to the** **suggestion** **in the Panel Report to endeavour to**
**develop "educated" correspondents in the customer services. The provision,**
**in the Fourth Framework Programme of targeted money to user services**
**which can be bid for on a competitive basis by the Centre should help to**
**alleviate the present insufficient funding in that area in view of the**
**magnitude of well founded requests and allow the JRC to show its mettle.**

**The administrative shortcomings, inherited from the overall Commission**
**system,** **are** **pointed** **out** **by everyone, especially** **those** **concerning**
**recruitment, opaque accountancy, lack of valuation of capital investment**
**and of intellectual property.**

**Recruitment of staff is found to be slow due to the tortuous and inefficient**
**system imposed on the Commission's scientific services as a whole. The Joint**
**Research Centre is a showcase for the Commission and the maintenance of**
**an irreproachable level of intellectual capacity and competence in its**
**scientific staff is vital. Effective recruitment becomes an even more pressing**
**point on taking into account the retirement bulge that will occur in the late**
**1990s.**

**Financial transparency and the demonstration of accountability** **have**
**become essential pre-requisites of any publicly financed organisation in**
**recent years and Visiting Groups do not seem to be entirely convinced on**
**that score.** **In** **the case of the JRC, the matter is complex.** **Financial**
**management** **must simultaneously meet the formal requirements of the**
**Commission's budgetary system and permit efficient operation of a research**
**organisation. Although improved as compared to some years ago, further**
**efforts in that direction could not fail to bring rewards through greater**
**perceived openness of the Centre in handling its affairs and thus the**
**removal of one cause of suspicion to the media and public.**

**Capital depreciation should be included more widely than at present as a**
**routine item in all costing calculations. It could be envisaged to apply an**
**amortisation period of 20 years to new buildings, some 15 years to large**
**equipment with perhaps 5 to 10 years to lesser items with more rapid**
**redundancy. The exclusion of accountancy for intellectual property serves to**
**hide the true value of JRC scientists and leads to undervaluation of the**
**Centre's potential. Both of these latter points carry the implicit criticism of**
**unfairness to the private sector in competitions.**

**On the scientific side, concern is rightly expressed over instances where**
**Groups feel that scientific staff is used for routine tasks which could be**

**?**

**better performed externally. New work** **should** **be rigorously assessed for**
**suitability before being accepted and the establishment of "bureaus"**
**avoided unless their establishment carries with it a substantial contribution**
**from the Centre's laboratories.**

**The various Reports** **seem** **to me to cohere well on these** **points.**

**5.** **The** **evolution** **of the JRC towards a more competitive regime has been**
**widely welcomed in the** **Groups'** **Reports. This is another step on a path**
**adopted for the reorganisation of the Centre in 1988, following the**
**recommendations of the Panel of Senior Industrialists led by Dr. Harry**
**Beckers. In most countries, competition in research organisations is now**
**considered to be a part of their natural development and the JRC has shown**
**a potential in that direction by making a good start with contract research in**
**a number of its Institutes. It is felt, however, that confidentiality of know-**
**how gained in performing contract research should be in general time-**
**limited,** **the norm could be some 3 to 5 years except where a longer time**
**span might be of special importance sucn as in particular defence-related**
**contracts (now within the** **JRC's** **remit), when special conditions might apply.**

**6.** **Most Visiting Groups comment on the problem of inter-Institute or pan-JRC**
**project management. With the forthcoming enlargement of the European**
**Union,** **it seems probable that there will have to be a corresponding increase**
**in the number of larger scale problems of concern to the Commission.**
**Environmental research has already** **seen** **a considerable broadening of its**
**horizons in the field of EU research activities, and in view of the**
**multidisciplinary nature of that type of problem, projects may well tend to**
**span the competence of more than one Institute. Examples of this already**
**exist with the Centre for Earth Observation or R&D on Nuclear Safeguards,**
**and such projects have an obvious subsidiarity value. The appointment of**
**senior persons with overall responsibility for such projects is important both**
**to ensure their smooth running but also as a clearly identified contact**
**person with the outside world.**

**7.** **In addition to the question of inter-Institute coordination, considerable**
**interest** **is** **shown in aspects of internal management of Institutes. Members**
**of Groups more concerned with the operation of research organisations**
**subjected to competitive pressures feel uneasy about the lack, here and**
**there,** **of established milestones or other targets by which group and**
**individual performance can be accurately assessed. In the relatively few**
**cases where adequate systems are operational, the Institute for Safety**
**Technology for instance, the resulting clarity and transparency of**
**presentation were much appreciated. Such a system should be considered to**
**be normal in a centre so much under the public microscope and would**
**simplify a** _**posteriori**_ **control by management.**

**8.** **To summarise, the tenor of the whole set of Visiting Group Reports, when**
**compared to the recommendations of the Panel of Senior Industrialists and**
**those of my own more recent Evaluation Panel, indicates that the Centre is**
**well on the way to becoming a recognised centre of excellence in many of its**
**areas of activity. In some areas, recognition is still lagging behind scientific**
**competence and the doors to the** **world,** **must be prised further apart.**
**However,** **I** **am extremely gratified to see that previous advice** **is** **being**
**effectively followed and that the word "OPENNESS" has entered the JRC**
**vocabulary.**

**5**

**Particular Points from Individual Reports**

**9.** **Institute for Reference Materials and Measurements** - **Geel**

The most important remarks here concern the staff which is dedicated and
able, the management which is caring and provides a clear lead; however,
more freedom to manage resources is recommended. Problems are
identified regarding the potential duplication of ongoing work in national
laboratories and the need for a more selective approach to try to reduce the
excessive number of small items undertaken so that the Institute can
concentrate on tasks falling within its field of excellence. Questions of
demarcation between chemical analysis tasks undertaken at Geel and at
Ispra also arise. Doubts are expressed on the operation of the BCR reference
materials sector where management and financial problems are foreseen.

**10.** **Institute for Transuranium** **Elements-Karlsruhe**

The need to maintain the Institute's strong nuclear interest and to remain a
centre of excellence was an important theme in the Report. There was
particular fear that the recruitment system employed by the Commission
might make the maintenance of an adequate staff of high competence
difficult to achieve. Attention was drawn to ensuring that exploratory
research would be directed towards potential developments of the
institute's core activities where possible. The high scientific level of Institute
staff was praised and there were doubts about the use of valuable personnel
on other sites to perform routine analysis. In contract research, the
Institute's performance is good, although its intellectual property is
somewhat undervalued, a point to be considered in setting up future
contracts. Commercial secrecy should be limited in time, both in the interest
of Member States and to ensure that the high quality of the work will
enhance the Institute's reputation to the fullest extent.

This Institute is the recognised centre of European actinide research.

**11** . **Institute for Advanced Materials (Materials) - Petten and** **Ispra**

The Institute has acquired a firm international reputation in its field with
good success in carrying out contract work and in network activities with
peer research teams in European and worldwide contexts. As with Karlsruhe
above, success in gaining research contracts has led to commercial secrecy
problems. Central rulings on this point could easily lead to a rigid and
stifling system, but the Marketing Adviser should be particularly concerned
with giving advice on this question. I am pleased to see that the Visiting
Group was able to endorse the Institute's long-term research plan based on
the study of interfaces in material. This is likely to continue the valuable
contribution made to EU industry by the Institute.

**12.** **Institute for** **Advanced** **Materials** **(The HFR** **Reactor)** **-Petten**

The Visiting Group gives the plant a clean bill of health and of safety,
recognising its excellent operational reliability and availability. However,
with the decline in nuclear R&D activities in most Member States, the nettle
of financial viability needs grasping. The possible use of a significant part of
the reactor for commercial purposes looks promising but implies that tariffs
are based on the cost of operation and major efforts to reduce staff and
operating costs need to be pursued with vigour. At the same time, it is clear

_**M>**_ **-6-**

that priorities would have to change with operating conditions geared more
to possible long term industrial clients than to scientific experimentation.
Such conditions would also place constraints on the Commission's freedom
to close the plant. An early decision is now required. The Report
demonstrates that the reactor has been exploited very successfully under its
R&D regime and that the quality of those involved is undeniable.

**13.** **Institute for Systems Engineering and Informatics - Ispra**

The Report is generally encouraging and demonstrates the great progress
achieved since the Panel review. However, the Institute is felt to be
hampered by the rules under which it operates, these being over rigid and
uncommercial. Most remarks apply equally to the rest of the JRC and some
of the more important ones are to be found in section 4. above. Again, the
importance of a clear and simple project management system is pointed out
and the difficulties of inter-Institute projects and informatics management
brought forward. A greater contact with industry is encouraged and the
adoption of commercial principles in bidding for third party contracts is
recommended.

**14.** **Environment Institute - Ispra**

The Report shows an institute producing a very high quality of scientific
research and service covering a rather over-extended number of items. A
rigorous selection of priorities could help with the problem. Once again,
one finds a warning against using hiçjhly qualified staff for routine research
and it is suggested that analytical services should be grouped together as a
separated service unit. The importance of a staff embracing a wide range of
disciplines in tackling environmental problems and the growing role of
computer modelling and international networks is emphasised and the
Institute is moving in those directions.

**15.** **Institute** **for** **Safety Engineering**        - **Ispra**

It is gratifying to find that the project control and management of the
Institute receive considerable praise, demonstrating the advantages of good
practice and transparency for the JRC. While recognising the reduction in
funding for research on nuclear safety, the Visiting Group was emphatic
about the importance of the Commission maintaining a viable activity in an
area where public concern was still high in many EU Member States,
especially in relation to Central and Eastern Europe. Research into the safe
handling of tritium and its waste is also essential if future public fears are to
be avoided when the first large-scale fusion power station demonstration
plant is undertaken. A further area I consider to be of public importance is
the developing research on certain aspects of industrial, particularly
chemical, plant safety. The Group pointed towards the development of
cooperative actions with other research bodies studying different aspects of
these problems to build a generic understanding of safety-relevant factors
in chemical manufacture, and to pursue this action further, recomends
hiring a few industrial chemists and chemical engineers. Work using the
new reaction wall facility to study structural behaviour under dynamic
loading is proceeding well and is very promising and the team at Ispra
already constitutes an important node in a large international network on
the subject. However, again there is a warning on not spreading efforts too
thinly over too wide an area of activity.

_**^M**_

**16.** **Institute for Remote Sensing** **Applications-Ispra**

Up to the present, about 50% of this Institute's activities has been concerned
with providing tools and methods for use by Commission sectorial services
responsible for policies concerning agriculture, anti-fraud, environment,
fisheries, regional aid and overseas development. The Commission's
Statistical Office is also an important client for the Institute's services. In
parallel with the above activities, and to improve their capabilities, research
is carried out on advanced methods of satellite data interpretation and its
integration with geographical data. The Centre for Earth Observation deals
with the management of the vast quantities of data emanating from
terrestrial satellites. The Institute's "customers", express a high degree of
satisfaction with the services rendered. With the development of Large
Applications Projects, new networks are being established in which the
Institute will play an important role. This relatively young and lively
Institute has become an appreciated partner by organisations such as the
European Space Agency, the IGBP and national space research centres. The
Visiting Group was concerned with the development of a viable strategy for
the Institute in harmony with national and international centres ancfwith
the transparency of its system for management information.

**17.** **Institute** **for** **Prospective Technological Studies - Ispra**

The Institute, set up in 1988 to provide scientific and technological forecasts
and advice in support of Commission services, is moving to a new site in
Seville during the latter part of 1994. The Report points out that the
Institute had not been able to meet the objectives originally foreseen for
reasons of lack of designated funding, but had evolved a successful role as a
scientific and technical advisory unit which should be further developed.
Under the Fourth Framework Programme funding is available to pursue the
missing "Observatory " function of the Institute. The Visiting Group praised
the Institute's small staff and considered the output to be "most
satisfactory". However, the problem of secrecy was again to the fore and
even hindered the detailed assessment of the unit's output by the Visiting
Group, which had to rely on interviews with clients! It is clear that although
certain advice to policy-makers might have to be on a need-to-know basis, in
general, the same time-restricted confidentiality system could be applied in
contracts and agreements as suggested for the rest of the JRC's activities.

**Conclusion**

18. In general, I perceive a very satisfactory response to the recommendations
made in the Panel report in 1991 and see steady progress since 1988 when
the JRC structure was changed and the Institutes were established. During
this time some of the rules of the game have changed, the introduction of
competition for example, and these pressures are making visible inroads
into the rigidity of Commission working methods and administrative
attitudes: further improvement is to be expected as this evolution continues
and is to be strongly encouraged.

19. It appears from this first round that the Visiting Group peer review system is
well suited to the task of evaluating the Centre on an Institute by Institute
basis. Reports show a good degree of coherence in unerringly identifying
areas for general improvement. An effort in the directions indicated can

**iz.** **-8**

**reap large rewards in efficiency and recognition. Concerning the latter**
**point,** **I** **have been interested to discover that** **many** **members of Groups**
**approached the Joint Research Centre with very little idea of either its**
**function or its capabilities except for a rather vague and invariably negative**
**reputation in national circles. On closer acquaintance, this attitude usually**
**changed to a strong interest in the Institute visited, which in a number of**
**cases has been followed by** **a** **proposal for collaboration in a specialised**
**research area. This demonstrates that although the** **JRC is** **a much more open**
**organisation than formerly, there** **is** **still an enormous potential for progress**
**in that sense.. Well directed efforts will certainly bring benefits to the**
**Centre, the European Union and its Member States.**

**20.** **Much has been achieved and the Board of Governors, Director General,**
**Directors and the JRC personnel have brought about a clearly recognisable**
**transformation of the Centre. With the Fourth Framework Programme, a**
**new phase is beginning under which the Joint Research Centre is given the**
**elements** **to complete this process to emerge as a modern and fully efficient**
**research** **organisation** **recognised in scientific and political circles as being an**
**effective and appropriate tool able to meet the needs of the Commission in**
**an evolving European Union.**

**»i**

**Annex**

**Visiting Group for the Evaluation of the Joint Research Centre**

**Scope of Visiting Groups**

**The** **chief** **scope of Visiting Groups is to perform evaluations, taking into account**
**the objectives as set** **out for** **the 1992-1994 period (in Council documents, relevant**
**papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to**
**provide recommendations on the proper course to be followed by the Institute**
**visixed (or a unit of an Institute) in view of the stated objectives, and to apply in**
**the evaluations,** **elements** **of the methodology proposed for research returns and**
**effectiveness.**

**The Visiting Group should:**

**perform an evaluation of all activities of the Institute/Unit against its stated**
**objectives, with particular emphasis on the impact the results of the work**
**has had on the user of this research, and** **the** **degree of collaboration**
**established with national research laboratories, as well as with industry;**

**assess the** **scientific** **quality of the work and the appropriateness of the work**
**and results** **for** **the Community;**

**judge all aspects of the efficiency and impact of the work, taking into**
**account the particular status of the JRC (its independence in the Community)**
**in comparison with similar national laboratories;**

**provide advice on the ongoing work on the basis of the evaluations and**
**assessments performed;**

**express recommendations for the future development of the** **Institute/Unit**
**with regard to the overall strategic plans for development of the JRC in the**
**medium-term and long-term.**

**Terms of Reference**

**To evaluate progress in performing work in accordance with the objectives**
**set out in Council** **Resolution 92/C118/03** **and Council Decisions 92/273/EEC,**
**92/274/Euratom and 92/275/Euratom, documents addressed to the Board of**
**Governors,** **thfc** **JRC's Annual Workschedules, budget etc., and to review**
**progress in the implementation of the recommendations of the last**
**Evaluation Panel.**

**2.** **To ensure that the Institute/Unit has effective mechanisms to set its local**
**scientific objectives, monitor progress towards those objectives and evaluate**
**outputs.**

**H**

**3.** **To review the various research activities to assess:**

**a)** **the scientific** **quality** **and** **productivity of** **the Unit/Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of** **funding** **and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management leadership within**
**the Institute/Unit.**

**5.** **To advise on whether the Institute/Unit has been making efficient, effective**
**and economical use of resources in carrying out its programmes and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's/Unit's external**
**scientific** **links,** **including relationships with research organisations, the**
**higher education sector, industry and government departments in the**
**Member States,** **as** **well** **as** **outside the European Community.**

**7.** **To examine the relationship of the Institute's/Unit's work in general, to the**
**mission of the Joint Research Centre and to** **its** **forward strategic planning.**

**8.** **To make recommendations and report to the Board of Governors of the**
**Joint Research Centre.**

**i** **y**

**Teddington, 2nd September,** **1994**

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR REFERENCE MATERIALS**

**AND MEASUREMENTS**

**a**

**CONTENTS**

**I.** **Introduction**

**II.** **General Observations**

**III.** **Scientific** **and Technical Activities**

**111.1** **Nuclear Physics and Measurements**

**111.2** **Reference Materials**

**111.3** **Exploratory Research**

**IV.** **Management of Research**

**IV.1** **Staff**

**IV.2** **Networking**

**IV.3** **Output**

**V.** **Recommendations**

**V.1** **General Points**

**V.2** **Particular Points**

**Annex** **I:** Members of the Visiting Group

**Annex II:** Terms of Reference of the Visiting Group

\A

**( ^**

**Report of the Visiting Group** **for** **the Evaluation of the Joint Research Centre's**
**Institute for Reference Materials and Measurements (IRMM), Geel, Belgium**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), members of the Visiting Group (see Annex I) spent 28-29 March 1994
at the Institute for Reference Materials and Measurements (IRMM)
(previously called the CerTtral Bureau for Nuclear Measurements) at Geel
reviewing the Institute's work and visiting its laboratories.

The objectives for the Institute are to be found in Annex I, "Scientific and
Technical Objectives", to Council Decision N° 92/273/EEC of 29th April
1992 published in the Offical Journal N° L141 of 23rd May 1992. The
relevant section is as follows:

"The Central Bureau for Nuclear Measurements will continue its activities
in both the nuclear and non-nuclear fields of reference materials and
measurements, notably in the following areas:

preparation, characterisation and certification of samples,

long-lived or stable isotope mass spectrometry,

nuclear data measurements and evaluation,

radionuclide metrology,

applied radiation techniques.

Research also in the growing application of these specialised techniques to
non-nuclear areas will capitalise on the existing experimental facilities
including the particle accelerators. Activities in support of consumer
protection policy will be continued ...".

These general objectives are concurrent with the JRC's mission and are
highlighted in the scope of the Visiting Groups. As set out in the terms of
reference (see Annex II), the Visiting Group should:

     - evaluate all activities of the Institute against its stated objectives,
with particular emphasis on the impact the results of the work has
had on the users of this research, and the degree of collaboration
established with national research organisations, as well as with
industry;

» assess the scientific quality of the work and the appropriateness of
the work and results Tor the Community;

    - judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the
Community) in comparison with similar national bodies;

» provide advice on the ongoing work on the basis of the evaluations
and assessments performed;

    - express recommendations for the future development of the Institute
with regard to the overall strategic plans for development of the JRC
in the medium-term and long-term.

**II.** **General Observations**

**During the 1992-1994 programme in course, the name of the Institute was**
**changed to the "Institute for Reference Materials and Measurements".**

**Activities were carried out under the Specific Programme on Measurement**
**and Testing of the Third Framework Programme for Community Activities**
**in the** **fieldof** **Research and Technological Development, support to other**
**Commission services and a small amount of contract research for external**
**third parties.**

**Work under the Specific Programme included:**

**neutron interaction data for standards, nuclear fission and fusion**
**technology;**

**radionuclide data and activities;**

**. high accuracy in isotope ratio measurements by mass spectrometry;**

**development for quality control analysis;**

**reference** **materials** **for** **elemental** **analysis,** **isotope** **ratio**
**measurements and reactor neutron dosimetry;**

**aerosol and surface reference materials;**

**samples and target preparation;**

**materials management on behalf of the "Bureau communautaire de**
**référence** **(BCR).**

**Support to other Commission services was at present concentrated on** **:**

**DG** **I** **(International** **Affairs-Nuclear** **Safeguards);**

**DG III (Internal Market);**

**DG VI (Agriculture);**

**DG XIII (Technology Valorisation);**

**DG XVII (ECSAM; Nuclear Safeguards);**

**Consumer Policy** **Service:** **(protection and promotion of consumer**
**interests).**

**Contract research for third parties:**

**This comprised a relatively minor activity at the present time.**

**The Visiting Group felt that in the time available it was not possible to**
**carry out a detailed audit but an overall appreciation was gained and valid**
**judgments were possible. The objectives of IRMM were generally taken as**
**given.** **A full assessment could not be made of whether the tasks or**
**organisation were fully justified or the best possible arrangement.**

**3-**

**It** **was noted that the** **change** **of institution name represented a major**
**emphasis that had occurred after the Council Decision of 1992 which**
**established the objectives of the Institute. The Group was uncertain about**
**the validity of the name change but felt the new name to be more**
**appropriate to the present IRMM research profile.**

**The management and staff provided a good level of information to brief**
**the visiting Group, were open, cooperative and tolerant in responding to**
**many questions and requests for more information. The Group** **wouldlike**
**to thank all those concerned.**

**In** **all areas we found dedicated and able staff who displayed a high level**
**of expertise. In some areas world class work was recognised and strong**
**input to international cooperation and networking was evident.** **The**
**culture was one of technical excellence but there was** **generally** **a**
**willingness to** **respond** **positively** **to** **change** **including** **increasing**
**competition.**

**The management was caring and attentive, providing a clear technical**
**and management lead whilst allowing staff to contribute. Some of the**
**JRC procedures and requirements were unhelpful and contributed to**
**delays in the timely deployment of resources. An example of the difficulty**
**is the slow recruitment procedure and the relative lack of freedom to**
**manage local resources against an agreed corporate plan.**

**III.** **Scientific** **and Technical Activities**

**111.1-** **Nuclear Physics and Measurements**

**In this original core activity area of the Institute, equipment and scientific**
**staff are found to be of a level to be expected in an international institute**
**of high repute. Connections through long-established networks with the**
**few other institutes in the world engaged in these activities were**
**appropriate and the teams were meeting the needs expressed by**
**international committees managing the accumulation of data in relation**
**to present and predicted needs.**

**At the same time, considerable creativity was being shown in attempting**
**to increase the application of methods and experimental equipment to**
**fulfil other needs. The presence of young post-doctorate fellows was a**
**considerable asset in that respect and the relationship with permanent**
**staff was demonstrably symbiotic.**

**The Group noted that the Linac GELINA, with its ultra-high resolution, was**
**being** **overhauled and updated. At present, in terms of resolution, the**
**machine is probably the best in the world.** **Moreover,** **in** **view of the**
**excellence of the very considerable installations such as the Linac and the**
**van de Graff accelerators, it is important that the investments they**
**represent should be utilised to the maximum. This may mean applying a**
**much greater effort towards their application to the advancement of**
**European industry, as well as their use for purely basic scientific ends. This**
**effort will require a much more intensive approach to possible partners or**
**clients** **and** **a** **clearer** **understanding** **of** **commercial** **and** **sales**
**considerations. However, this is unlikely to result in more than a 10% use**
**for** **such purposes.**

_**Z**_

_**Z^**_ **-4-**

**Some** **four years after the completion of the refurbishement of the**
**GELINA it would be wise to review its work and situation in scientific,**
**economic and organisational** **terms.**

**The quality of ongoing work is not in doubt but in the changing situation,**
**attention must be paid to** **the** **potential added value of the intellectual**
**property available. Amongst other things, the development of the use of**
**Van de Graff for** **Rutherford scattering** **as an analytical tool seems**
**particularly promising.**

**The question of subsidiarity does not seem to be of great importance in**
**this area,** **as** **such equipment is either unique or rare in member States,**
**particularly if the accompanying expertise is taken into** **account.** **At the**
**same time, the work in this field is seen to be important and regulated**
**and monitored externally, although the bodies concerned do not handle**
**the funds and are therefore not really "customers" in the classical sense;**
**this implies that a more critical internal progress review system may** **be**
**needed in the future.**

**An area where some members of the Visiting Group had some**
**reservations was in data processing and informatics where it was felt that**
**streamlining and the adoption of a more modern and distributed**
**approach would be more appropriate.**

**III.2** **Reference Materials**

**This item includes analytical science, mass spectrometry and sample**
**preparation.**

**Facilities in this area were also found to be excellent, particularly where**
**the clean laboratory and mass spectrometry facilities were concerned**
**which were of** **worla** **standard and the analytical laboratories which were**
**very well equipped.**

**Staff were extremely competent and include scientists of international**
**repute. The part played by Geel in the redetermination of the Avogadro**
**number has brought international recognition with implications for the**
**traceability of chemical standards.** **Apart from the subject of this highly**
**networked activity, there is spin-off into the improvement of isotope**
**dilution techniques, an important technique for analytical purposes.**

**Chemical metrology is important and more effort should be concentrated**
**on the improvement of separation and extraction techniques, as well as**
**the problems of reliable sampling and homogenisation. This work should**
**also be networked with and support other standards' laboratories in**
**Europe and, where appropriate, elsewhere.**

**Actinide reference materials are the reference** **against** **which fissionable**
**materials are measured for international safeguards. Geel provides an**
**important part of these sources, both in Europe and elsewhere.**

**The Institute has also initiated and coordinates the** **International**
**Measurement Evaluation Programme** **(IMEP).** **A large worldwide network**
**of laboratories participate in this action and their acceptance of IRMM's**
**role as coordinator stands witness to its reputation as a centre of scientific**
**excellence in that area.**

**-5-** **2-3**

**The Group was concerned about the economics of the reference materials**
**action performed on behalf of the Community Reference Bureau (BCR).**
**We understand that the staff involved would be paid from the Institute's**
**budget but that sales will have to provide the funds needed to produce**
**new reference materials and for the provision and re-certification of**
**replacements for time-expired or exhausted stocks of existing materials.**

**In moving** **into a competitive situation, a very careful analysis by business**
**experts will be particularly important if the Union is to be provided with a**
**permanent, reliable and cost effective service. To obtain such advice,**
**detailed costing of staff, materials, operational procedures, storage and**
**investments will certainly be needed. Consideration should be given to**
**contracting** **out the provision of this service to a commercial organisation.**

**Furthermore,** **quality assurance under an internationally** **recognised**
**standard such as ISO 9000 will be essential as many of the organisations**
**using the products are already accredited or are becoming so.**
**The Group was concerned that Geel should concentrate on its core area of**
**very high quality reference materials and measurements and not use**
**expensive resources of staff and equipment for routine work.**

**A clearer definition of work carried out at Ispra and that performed at**
**Geel seemed to be needed as was greater selectivity in jobs accepted from**
**other Commission services where cheaper alternatives could sometimes be**
**found.**

**III.3** **Exploratory Research**

**In examining on going exploratory and discretionary research at Geel, the**
**Visiting Group noted that the two most important projects were linked to**
**the Institutes activities in different ways. The "Radiation Physics" project**
**may well widen the application of GELINA and can lead to more physicists**
**wishing to use the facility. On the other hand, the investigation of** **protein**
**and peptide components of metallothioneine is more concerned** **with**
**work falling under Support to the Consumer Policy Service.** **Further**
**development of such work on applied and basic biochemistry at IRMM has**
**implications on the long-term forward planning of the Institute and**
**should be a matter for very careful review.**

**IV.** **Management of Research**

**IV.1** **Staff**

**With the low rate of staff turnover it was felt to be extremely important**
**to maintain a good flow of postdoctoral and other research fellows and**
**visiting scientists in the Institute.**

**The level of expertise was considered to be very high at senior staff level**
**and their replacement will require careful planning at retirement time. A**
**more flexible recruitment system able to operate more swiftly and a**
**consistent staff training and career development policy would help to**
**meet the changing situation in research and establish the desired staff**
**profile in the years to come. (Some 5% of available time spent on staff**
**development would be reasonable.)**

_**m**_ **-6-**

**Members felt that it would also be useful to examine the use of short-term**
**appointments or long-term attachments, of three to five years, with**
**guaranteed return to their institute of origin to increase the throughput**
**of scientists at IRMM and enhance knowledge transfer.**

**IV.2** **Networking**

**Tasks are well defined in key areas but in those where qualitatively new**
**venture have been initiated definitions are less clear and not yet**
**integrated into strategic** **orientations.**

**Further efforts applied to defining the Institute's tasks and its effective**
**boundaries would considerably help with education and training and with**
**maintaining morale and interest at all staff levels.**
**The nature of the tasks performed at Geel has naturally led to the**
**development of networks and such actions are considered to be an**
**important means of information dissemination, scientific exchange and**
**generally ensuring the Institute's place in the European Union** **andWorld**
**at large. This** **is** **strongly encouraged.**

**A better feeling amongst scientific managers for hourly staff costs and**
**real availability would help with planning in a more competitive world.**

**IV.3** **Output**

**This was seen to take diverse forms such as data, reference materials,**
**certificates, as well as the more traditional ones of articles in learned**
**journals, presentations at conferences etc. Taken overall, the quality and**
**quantity are considered to be satisfactory and in many cases represent**
**significant contributions to their respective fields of endeavour. A share**
**of these has corne from the young scientists.** **However, continuous**
**performance monitoring can help to improve the value of the JRC as**
**perceived by scientists and managers in Member States and efforts to**
**increase output can only result in a wider and improved reputation.**

**The temptation to demonstrate willingness by taking on work of an**
**unsuitable or routine nature should be resisted.**

**Recommendations**

**With these recommendations, the Visiting Group would like to contribute**
**to the further evaluation and development of the activities at IRMM.** **The**
**points indicated should be taken into consideration at the various levels of**
**decision making and management. The Visiting Group notes that some**
**elements** **indicated** **in these** **recommendations** **are** **already** **being**
**employed in some form and it also underlines that in the implementation**
**of such recommendations account has to be taken of the specifics of the**
**various research areas at IRMM.**

**V.1** **General Points**

**There is a need for greater clarity about the mission of IRMM, having**
**regard for the needs of the EU, the mission of other JRC Institutes,**
**subsidiarity and the special capability at IRMM.**

**2** **y**

**High level performance measures should also be set and monitored.**

**There is a need for a robust scrutiny of project proposals to ensure they**
**satisfy a priority EU need and that the IRMM is best placed in terms of cost**
**and technical capability to deliver the work. Such scrutiny should take**
**place during the preparatory phase of programme generations. Effective**
**management and value for money depend critically on the existence of**
**independent customers for IRMM work who also have responsibility for an**
**input to the framework programme work commissioned from Member**
**States.**

**The spread of technologies and** **techniques** **at IRMM should be reviewed**
**with a view to reducing the spread** **ana** **increasing the depth in selected**
**areas.** **In particular the roles of IRMM and Ispra in the field of chemical**
**analysis should be reviewed and rationalised to achieve greater economy**
**of scale.**

**The level of laboratory facilities is high compared with staff numbers. This**
**applied to both big facilities such as the linear accelerator and a vast array**
**of smaller pieces of equipment. A more business-like approach is required**
**which starts with knowledge of the true cost of ownership.** **By a**
**combination of better focussing of work, down sizing, where appropriate,**
**and business growth in selected areas it would be possible to increase the**
**return on capital.** **It** **is recommended that a programme to assess true**
**costs be initiated alongside an assessment of market opportunities and**
**business development.**

**As a more customer orientated programme develops more attention will**
**need to be paid to project management with systems in place for project**
**planning,** **cost control, milestone setting and monitoring.**

**The proposed review of QA needs is welcome and consideration needs to**
**be given to ISO 9000, ISO 25, TQM and benchmarking approaches. It is**
**recommended** **that** **benchmarking** **be** **carried** **out** **for** **selected**
**management** **and** **technical** **functions** **against** **other** **comparable**
**organisations in national** **government** **ownership and privately owned.**
**Benchmarking should provide information such as cost of work, cost of**
**facilities, output per scientist, cost per scientist.**

**Many of the facilities represent major European resources which should**
**be made more widely available to Member States.** **Work of EU and**
**Member States' interest should be facilitated free of charge against a**
**budget held by DG XII and within current levels of funding.**

**Some of the work carried out was in competition with that undertaken by**
**Member States and in some instances there is a danger of heavily**
**subsidised IRMM activities being in direct competition with more cost**
**effective commercial work in Member States. Examples of this are the**
**chemical method development and chemical reference materials work**
**carried out in support of EU agricultural, environment and consumer**
**safety policy. It is recommended that review of these activities be carried**
**out to determine whether it is possible to deliver this work in a more cost**
**effective way within Member State institutions and if appropriate to**
**advise on the resolution of competitive and subsidiarity conflicts.**

_**jl**_ **-8-**

**V.2** **Particular** **Points**

**The move towards a competitive model for the Institute is**
**applauded both as a means of increasing scientific vitality and**
**awareness of staff, but also as a demonstration of the Institute's**
**ability and usefulness.**

**Effort must be applied to increasing, where possible, contract**
**research and** **services** **to industry and other clients.**

**The cost** **effectiveness** **of** **facilities should** **be established and those**
**that need to be** **retained** **should be marketed in government and**
**industry on an international basis.**

**[Mechanisms should.be put in place to establish the full economic cost](http://should.be)**
**of work and comparisons of cost made with similar work in other**
**organisations.**

**Independent customers should be established who are responsible**
**for defining the requirements and commissioning work in the most**
**cost effective way. The customers should also be responsible for**
**work commissioned from Member States within the Framework**
**Programme.**

**Where possible, two-way temporary staff exchanges should be set**
**up** **as** **a means of cross-fertilisation.**
**Rigorous selection should be applied to work carried out by the**
**Institute and a clear forward plan adopted. Work division between**
**Geel and other** **JRC** **Institutes Should be established and recognised.**

**The roles of IRMM and Ispra in chemical analysis should be reviewed.**

**Chemical method development and associated reference materials**
**development** **in support of agricultural, environmental** **and**
**consumer safety policy should be reviewed to determine whether**
**this work could be carried out more** **effectively** **within Member State**
**institutions.**

**Commercial advice should be sought in a number of areas and**
**particularly for the management of reference materials where**
**doubts about future profitability exist.** **Contracting out to a**
**commercial organisation should be considered.**

**International** **cooperations on chemical traceability** **should** **be**
**continued but work should be extended to** **include** **sampling and**
**homogenisation of analytical samples.**

**More freedom to manage resources against an agreed corporate**
**plan should be given to the Director. The rigidity of administration**
**procedures should** **be reduced,** **especially** **where** **delays** **in**
**recruitment are concerned.**

**Annex** **I** **2f—**

**Visiting Group to the Joint Research Centre's Institute for**
**Reference Materials and Measurements (IRMM) - Geel**

**Members**

**Dr. Maria Grazia DEL MONTE,**
**Director,**
**National Centre for Reference Materials,**
**Via di Castel Romano 100-102,**
**1-00129** **Rome.**

**Dr.** **Robert** **KAARLS,**
**Director,**
**Netherlands Measurement Institute,**
**P.O.** **Box** **654,**
**NL-2628 VK Delft.**
**(Schoemakerstraat 97,**
**NL-2600AR** **Delft.)**

**Dr.** **Bernard** **KING (Chairman),**
**Director,**
**Analytical Quality,**
**Laboratory of the Government Chemist,**
**Queens Road,**
**Teddington,**

**UK-TW11** **0LW Middlesex.**

**Prof.** **Thomas** **SCHOENFELI),**
**Institute of Inorganic Chemistry,**
**University of Vienna,**
**Wahringerstrasse** **42,**
**A-1090** **Vienna.**

**Secretary**

**Mr. SamuelJ. LLOYD,**
**Avenue Reine Elisabeth 8,**
**B-1330Rixensart.**

**Tel. + 39-6-505.52.61**
**Fax. +39-6-505.02.50**

**Tel. + 31-15-69.15.07**
**Fax. + 31-15-61.29.71**

**Tel. + 44-81-943.76.00**

**Fax. +44-81-943.75.93/943.27.67**

**Tel.** **+43-1-403.05.44/310.23.85**
**Fax. +43-1-403.05.44**

**Tel. +32-2-295.40.48** **(office)**
**+ 32-2-654.09.44 (home)**
**Fax. +32-2-295.01.46**

_zi_

**Annex** **II**

**Visiting Group** **for** **the Evaluation of the Joint Research Centre's**
**Institute for Reference Materials and Measurements (IRMM)** **-** **Geel**

**Scope of Visiting Groups**

**The chief scope of Visiting Groups is to perform evaluations, taking into account**
**the objectives as set out for the 1992-1994 period (in Council documents, relevant**
**papers for the** **Board** **of Governors, JRC Annual Workschedules, budget,** **etc.),** **to**
**provide recommendations on the proper course to be followed by the Institute**
**visited in view of the stated objectives, and to apply in the evaluations, elements**
**of the methodology proposed for research returns and effectiveness.**

**The Visiting Group should** **:**

**perform an evaluation of all activities of the Institute against its stated**
**objectives, with particular emphasis on the impact the results of the work**
**has had on the user of this research, and the degree of collaboration**
**established with national research laboratories, as well** **as** **with industry;**

**assess the scientific quality of the work and the appropriateness of the work**
**and results for the Community;**

**judge all aspects of the efficiency and impact of the work, taking into**
**account the particular status of the JRC (its independence in the Community)**
**in comparison with similar national laboratories;**

**provide advice on the ongoing work on the basis of the evaluations and**
**assessments** **performed ;**

**express recommendations for the future development of the Institute with**
**regard to the overall strategic plans for development of the JRC in the**
**medium-term and long-term.**

**Terms of Reference**

**To evaluate progress in performing work in accordance with the objectives**
**set out in Council Resolution** **92/C118/03** **and Council Decisions 92/273/EEC,**
**92/274/Euratom** **and 92/275/Euratom, documents addressed to the Board of**
**Governors, the** **JRC's** **Annual Workschedules, budget** **etc.,** **and to review**
**progress in the implementation of the recommendations of the last**
**Evaluation Panel.**

**To ensure that the Institute has effective mechanisms to set its local scientific**
**objectives, monitor progress towards those objectives and evaluate outputs.**

**3 P**

**3.** **To review the various research activities to assess:**

**a)** **the scientific quality and productivity of the** **Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of funding and other** **constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external** **customers,** **when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management leadership within**
**the Institute.**

**5.** **To advise on whether the Institute has been making efficient, effective and**
**economical use of resources in carrying out its programmes and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and** **equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the higher**
**education sector, industry and government departments in the Member**
**States,** **as** **well as** **outside** **the European Community.**

**7.** **To examine the relationship of the Institute's work in general, to the mission**
**of the Joint Research Centre and to its forward strategic planning.**

**8.** **To make recommendations and report to the Board of Governors of the**
**Joint Research Centre.**

_**M.**_

**Stockholm, 28th January 1994**

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR ADVANCED MATERIALS**

3 ^

**3** _**I**_

**CONTENTS**

**I.** **Introduction**

**II.** **General Observations**

**III.** **Scientific** **and Technical Areas**

11.1 Materials in Extreme Environments

11.2 Reliability of Life Extension
11.3 Surface Modification Technology

11.4 Fusion Materials

11.5 Non-Destructive Testing
.6 The Cyclotron

II.7 Materials Information

**IV.** **Prenormative** **Research**

**V.** **Exploratory Research**

**VI.** **Networking**

**VII.** **Publications**

**VIII.** **Future Scientific Trends**

**IX.** **Management**

**X.** **Recommendations**

**Annex** **I:** Members of the Visiting **Group**

**Annex II:** Terms of Reference of the Visiting Group

**page** **1**

**pages 1-3**

**pages 3-5**

page 3
page 3
page 3
page 4
page 4
page 4
page 4-5

**page** **5**

**pages 5-6**

**page** **6**

**page** **6**

**page** **6**

**page** **7**

**page 7-8**

3</

_**iï**_

**Report of the Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute for Advanced Materials, Petten (The Netherlands) and Ispra (Italy)**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), members of the Visiting Group (see Annex I) spent the 14-17
December 1993 at the Institute for Advanced Materials (IAM) at Petten and
Ispra reviewing the Institute's work and visiting its laboratories.

The objectives for the Institute are **to** be found in Annex I of Council
Decision 92/273(EEC) and 92/274/Euratom of 29 April 1992.

The main objectives are to contribute to enhancing European industrial
competitiveness and the safety of components and structures, relevant
mainly to the energy, transport, environment and manufacturing sectors,
through advances in the understanding of the basic properties, the
processability and engineering performance of materials.

These general objectives are concurrent with the JRC's mission and are
hiqhlignted in the scope of the Visiting Groups. As set out in the terms of
reference (see Annex II), the Visiting Group should:

      - evaluate all activities of the Institute against its stated objectives, with
particular emphasis on the impact the results of the work has had on
the users of this research, and the degree of collaboration established
with national research organisations, as well as with industry;

      - assess the scientific auality of the work and the appropriateness of
the work and results Tor the Community;

      - judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the
Community) in comparison with similar national bodies;

      - provide advice on the ongoing work on the basis of the evaluations
and assessments performed;

      - express recommendations for the future development of the Institute
with regard to the overall strategic plans for development of the JRC
in the medium-term and long-term.

**II.** **General Observations**

The Institute's functional structure is set out below. The High Flux Reactor
(HFR) is not included in this Report, having already been visited by experts
in research reactors and the subject of a recent report. The structure of the
Institute is as follows:

High Flux Reactor Petten **(separate report)**
Materials Processing and Engineering Petten
Materials Characterisation Petten
Non-Destructive Testing Petten (recently transferred
from Ispra)

**X**

**Functional Materials Development** **Ispra**
**and Cyclotron**
**Materials Performance and Reliability** **Ispra**

**The Visiting Group examined the Institute in detail and found the work and**
**scientific output to be very good or excellent in most areas and of a** **truly**
**international standard. The targeting of the work was good and staff**
**showed a clear understanding of objectives.**

**The Institute is well managed. However, the Visiting Group believes that**
**the Director should be given more responsibility for running the Institute,**
**particularly in the recruitment of staff, and that consideration should be**
**given to exempting the Institute from some internal European Commission**
**procedures.**

**The Institute is undoubtedly expensive, with high overheads on its**
**research.** **However, the Visiting Group is convinced of the need for a**
**European** **institute of this type in the key strategic area of advanced**
**materials. Given this need, the Visiting Group accepts that there are many**
**various reasons for the high overheads, including the need to pay**
**competitive salaries, the inherent expenses of running a high flux reactor**
**and a** **cyclotron,** **and so on. The Visiting Group saw no evidence of waste.**

**The Visiting Group believes that funding of the Institute should move**
**towards a "core plus competitive" funding model. Contract work for third**
**parties, mainly industry, has built up rapidly in the last few years and is at**
**present running at about 8% of the total budget.** **The Visiting Group**
**believes that the aim should be to increase this to 10% over the next few**
**years.** **Such contract work is not only useful financially, but it also builds up**
**real links with industries in the Member States.** **The Visiting Group**
**welcomed the proposal at** **present** **under discussion at the political level to**
**allow the JRC to participate in Framework Programme schemes, for**
**example** **BRITE/EURAM.** **It is recommended that the aim of the Institute**
**should be to obtain about 20% of its funding competitively from**
**Framework Programme schemes over the next few years. Apart from** **the**
**increased competitive funding element this would introduce, the Visiting**
**Group believes that bidding for, and participation in, collaborative**
**research** **through** **Framework** **Programme** **schemes** **would** **greatly**
**strengthen the Institute's links with universities, research institutes and**
**industries in the Member States and would be wholly beneficial.**

**Orientation was greatly aided by the extensive use** **of** **colloquia, round**
**table discussions and workshops, with** **strong** **industrial representation,**
**when setting up programmes and activities, thus creating a consensus in**
**the materials science community. In the same way, seminars, conferences,**
**expert meetings** **and** **publications in relevant international journals**
**ensured that scientists were kept abreast of progress.**

**The detailed internal annual review of all areas of work and project control**
**were felt to be important factors in demonstrating and maintaining the**
**level of excellence.**

**The Institute is** **considered** **to hold the lead position in the special area of**
**research on High Temperature Materials, particularly for its work on**
**corrosion/mechanical properties interactions both for its expertise and for**
**its** **equipment.** **This should be built on to further improve its image and to**
**play key European roles by:**

3- **^**

exchanges of staff;
network establishment and participation in Framework Programme
schemes;
giving impartial advice on standards;
encouragement of the introduction of advanced materials to improve
competitiveness.

**III.** **Scientific and Technical Areas**

111.1 Materials in Extreme* Environments

In the study of behaviour of industrial materials in extreme service
environments, the Institute at present occupies an internationallyacknowledged leading position.

The Visiting Group considers that the area is still an important one and that
this work should continue to be emphasised. The Institute's well deserved
reputation should be built on by placing more weight on aspects of stress
analysis with the recruitment of an applied mechanics expert as soon as
feasible. More emphasis could also be placed on computer modelling and
integration with engineering design. Concerning the first aspect, this
should preferably be handlecfby setting up networks with universities and
research institutes. With regard to the second, this would profit from close
interfacing with engineering design teams.

The study of metallic, ceramic and composite materials for high
temperature applications is of considerable importance to industry and
networks and joint research programmes with universities, industrial
research and government research organisations should be encouraged.

111.2 Reliability of Life Extension

The difficult and expensive experimental activities in this area show good
results and some useful spinoffs to industry. However, the core area of this
research should concentrate on developing more advanced procedures for
assessing operational damage. Other aspects would profit from more
intensive collaborations with external research bodies.

111.3 Surface Modification Technology

The Visiting Group was satisfied by the very rapid establishment of this area
of activity at the Institute. After a short time useful results are already
being obtained and the major equipment is functional. Work should now
be focused on contributing to high temperature industrial applications of
these techniques and this perspective should be introduced as early as
possible to these activities with emphasis on the production of data which
will stimulate increased industrial use of the techniques. The Group did not
feel that wear studies should be developed in the Institute as the subject
was highly researched in many other places and was somewhat out of line
with other Institute topics. A networking approach could help to avoid
duplication of equipment and scientific effort.

_**bi**_ **- 4 -**

**111.4** **Fusion Materials**

**Although a small area of activity, very good and important work had been**
**done up to the present on materials screening and selection in connection**
**with the Community Fusion Programme. The priority given to this work**
**must depend on the needs expressed by the programme managers as the**
**Visiting Group itself felt unable to make a valid judgment on this highly**
**specialised materials application.** **However, Members would warn the**
**team against entering the field of tritium hermetic ceramic composite**
**structures without adequate industrial, engineering and financial support**
**as the subject was a vast one with strong dependence on proprietary know-**
**how whicn might not be easily available.**

**An adequate data base on materials for fusion was in any case of high**
**importance.**

**111.5** **Non-Destructive Testing**

**This activity had been extremely useful in the past** **(PISC** **programme) and**
**was at present being transferred from** **Ispra** **to Petten. The Visiting Group**
**considered this to be an advantage as the previous nuclear orientation of**
**the team could** **now** **be turned towards high temperature materials**
**applications where** **a** **close** **collaboration** **with scientists working on the**
**assessment of creep damage and remaining lifetime in high temperature**
**plant could lead to great savings and an improvement in safety. This area**
**should be** **vigorously** **pursued and work extended to include ceramics and**
**composites in the future. To advance as rapidly as possible it was essential**
**that the entire team and equipment be reunited at Petten without delay.**
**The task was a very challenging and important one with very considerable**
**industrial applications, both on bulk materials and coatings.**

**Plant in the energy and process industries was ageing and would come to**
**rely more and more on reliable Non-Destructive Evaluation to assure**
**continued operation and the amount of work on the topic carried out in**
**universities was limited emphasising the work at** **Petten.** **The Institute**
**should,** **however, be encouraged to collaborate** **in** **this research with**
**universities and other research bodies.**

**111.6** **The Cyclotron**

**The main objective of this equipment was the study of damage in materials**
**destined for fusion applications. Unfortunately, there were few other uses**
**in the Institute's other activities and external users had been sought to**
**enable the installation** **to** **remain economic.** **However, use by other**
**research programmes did not appear to be forthcoming and alternative**
**solutions should be examined.**

**As the** **remaining** **work is of high quality and importance to fusion**
**materials research, if the Cyclotron becomes unavailable the possible use of**
**other facilities should be examined.**

**Materials Information**

**The establishment of data for advanced materials is an essential part of**
**their introduction into industrial use. Easy access to comprehensive data is**
**vital for industry.**

_**3J**_

Due to the developments during the last few years the HTM Databank is
now in a very attractive form. To ensure that a much wider use of the
databank is achieved, two actions should be taken :

i) most of the information in the system (for example all data which is
older than five years) should be declassified to research organisations
in the EU and EFTA countries. The confidential data has, to a large
extent, been generated in the COST framework and other European
programmes with financial contributions from the national
governments anyway;

ii) the Institute should participate in European programmes like the
European Collaborative Creep Committee where valuable use of the
data can be found.

Data banks are not in general commercially viable and often contain
confidential and restricted data. The JRC, with its central and neutral

C osition, ase which is indeed closely connected to the Institute's research. is in an ideal position to develop and supervise this type of data

A number of new data bases appear to be needed, not only to handle data
like the HTM Data Bank, but also to be a source of constitutive equations.
There is considerable need to be found for this type of database in thermal
fatigue research. To save time and avoid duplication of studies, such a
databank could preferably be set up as a network in the sense that
participating institutes have their own data banks which they fill with data
and this data is then exchanged between the partners to create a true
European system. This approach has successfully been utilised for
Thermodata Europe.

IV. Prenormative Research

The Institute should develop further prenormative research. Although not
at the cutting-edge of science and technology, this type of research ensures
that results can really be used in practice. Work should be concentrated in
areas where there is a realistic prospect of application. In this respect, the
proposed European Prenormative Research Associations (EPRA) and
EPRA-M (for Materials) could prove to be a realistic means of assessing
European requirements in this kind or research and providing a suitable
forum. The Institute has demonstrated its possibilities in this area and this
will hopefully lead to appropriate and usable standards. Such work can
best be pursued as an international exercise and the Institute, by its nature,
is highly suited to such activities. The Institute should be encouraged to
have a coordinating role in the development of materials standards in
Europe, working closely with national standards laboratories. Work in this
area has to be considered of high importance when comparing the
situation in Europe with that in the U.S.A., or even increasing prenormative
efforts in Japan.

**Exploratory Research**

The Visiting Group was impressed by and strongly supports the exploratory
research programmes of the Institute. The Group considers exploratory
research to be an important means of stimulating scientists to feed new

_**Ho**_

ideas into mainline activities and to explore alternative approaches and
new techniques suitable for application within the Institute.

Such activities should be frequently reviewed and either transferred to
mainline research or terminated. To emphasise its exploratory character, it
is expected that many projects would be terminated at an early date to
make room for more promising ideas, only the best being retainea.

**VI.** **Networking**

The Visiting Group was pleased to see a positive attitude to networking in
the Institute. In its main fields, the Institute could consider itself to be the
natural leader (due to its competence) of networks, but it should be
realised that at present, when research funding is limited, resources would
have to be provided by the Commission. In that respect, access to compete
in Framework Programme Shared Cost Actions might be handled on a
collaborative level with other research units, thus forming the basis of a
network. In addition, if the Institute was a full partner in BRITE/EURAM
programmes it would greatly strengthen collaboration between the
Institute and leading laboratories in the Member States.

Members of the Group pointed out that exchanges should be two-way and
that mobility must be expected from JRC staff as well as from their
partners. This would be of great benefit for knowledge transfer and
acquiring and passing-on new laboratory techniques. Visits should include
established scientists which would further increase the Institute's visibility
in the outside world.

The possibility of PhD students dividing their time more equally between
the JRC and their universities should be examined (six months in each per
year). JRC staff could aid in the setting-up of materials laboratories
elsewhere in Member States.

**VII.** Publications

The Group was pleased to see a good level of 43 articles published in
appropriate international scientific journals during the year, and
favourably impressed by the level of 52 papers contributed to international
conferences of which 15 were invited. These figures demonstrate the
interest shown in the Institute's work in the international arena. There
were also some 22 articles in European journals and 28 papers to European
conferences.

**VIII.** **Future Scientific** Trends

The Visiting Group strongly agreed with the Director's plan for the Institute
that interfaces in materials should be the major theme. The structure and
properties of interfaces are critical to the performance of a wide range of
materials, particularly high temperature materials, and interfaces are not
well understood.

**7-**

**IX.** **Management**

**The Institute was well managed, with scientific staff enthusiastic and**
**spending a high proportion of theirtime on scientific and technical tasks.**

**The systematic use of workshops in assessing new possibilities and the**
**follow up audits of activities was shown to Be effective.** **However, the**
**Institute should not be overburdened with irrelevant committees and in**
**that connection Members of the Group did not feel that the local** **Scientific**
**Committee served any useful purpose.** **Scientific or managerial advice**
**could be better obtained from academic and industrial scientists of**
**requisite experience and from proven managerial experts.**

**Ideas from staff members would seem to be adequately catered for by the**
**possibilities opened by exploratory research activities.**

**The ratio of support staff to scientists was felt to be rather high in**
**comparison with universities and many industrial laboratories.**

**The transfer of responsibility to local Directors was recognised and**
**encouraged. In that respect, maximum transparency of overall budget and**
**staff matters was essential.** **However, much was to be desired in the**
**recruitment of staff which appeared too slow and rigid for a scientific**
**institute.**

**In view of the relatively high man/year costs, the Institute must concentrate**
**on its core activities wnere its exceptional results** **were vital** **to a successful**
**reaction** **from** **potential clients and scientific collaborators.**

**X.** **Recommendations**

**•** **The Director should be given more managerial** **responsibility,**
**particularly in staff appointments.**

**•** **The Institute should move towards a "core plus competitive" funding**
**model.**

**•** **The Institute should aim at obtaining 10% of its funding from**
**external contract work** **from** **industry over the next few years.**

**•** **The Institute should be allowed to participate fully in Framework**
**Programme schemes such as** **BRITE/EURAM.** **It should obtain about**
**20%** **of its** **funding** **competitively** **from** **such schemes over the next few**
**years.**

**•** **Work on advanced high temperature materials should continue as the**
**main core activity.**

**•** **An applied mechanics expert should be added to the team.**

**•** **Transfer of non-destructive testing staff from Ispra to Petten should**
**be completed without delay.**

**•** **Surface modification work needs clearer alignment towards industrial**
**needs.**

**v/**

**8**
_**H**_ _**[llr ]**_

**Computer modelling should be** **increased** **where appropriate and**
**maximum use made of networking with other organisations to meet**
**these needs.**

**A programme of exchanges of scientists with other organisations**
**shoulcfbe** **set up.** **JRC** **staff should be included in exchanges.**

**The prenormative area should be developed** **as** **a typical** **JRC** **activity.**

**The JRC should be source of impartial advice and expertise on**

**materials standards.**
**Publication in the most respected scientific journals** **should** **be**
**encouraged.**

**An early decision should be taken on the future of the Cyclotron.**

**Open** **access** **data bases should be set up where possible.**

**The long-term research plan of interfaces in materials is a highly**
**suitable path for the Institute to follow.**

**Annex** **I** **"O**

**Visiting Group to the Joint Research**
**Institute for Advanced Materials** **(IAM-Mat)** **- Ispra and Petten**

**Members**

**Prof.** **Claude BATH1AS,**
**Department of Metallurgy,**
**Conservatoire National**
**des Arts et Métiers (CNAM),**
**292 Rue St. Martin,**
**F-75003 Paris.**

**Tel.** **+33-1-40.27.23.22**
**+ 33-1-40.27.26.33**
**Fax. +33-1-40.27.23.22**

**(Professor of Mechanical Engineering, Head of Institue for Technology and**
**Advanced Materials at CNAM, Fellow of the American Society for Materials,**
**Voting Member of the American Society for Testing and Materials)**

**Prof.** **Colin** **J.** **HUMPHREYS,** **Tel.** **+44-223-33.44.57**
**Head of Department and** **Goldsmiths'** **+44-223-33.44.58**
**Professor of** **Materials'** **Science,** **Fax. +44-223-33.44.37**
**Department of Materials'**
**Science and Metallurgy,**
**Pembroke Street,**
**University of Cambridge,**
**UK-Cambridge CB2** **3QZ.**

**(Member of Academia Europaea, Vice-President of the UK Materials Institute,**
**Director of Rolls-Royce University Technology Centre)**

**Prof.** **Rolf** **SANDSTRÔM** **(Chairman),**
**Swedish Institute for Metals Research,**
**Drottning Kristinas** **vàg** **48,**
**S-11428** **Stockholm.**

**Tel.** **+46-8-790.83.20**
**Fax. +46-8-20.31.07**

**(Professor of Applied Materials** **Technology** **at the Royal Institute of Technology,**
**Vice-Dean of the School of Mechanical and Materials Engineering and Chairman**
**of the Materials Centre at the same university)**

**Secretary**

**Mr. Samuel J. LLOYD,**
**Avenue Reine Elisabeth 8,**
**B-1330Rixensart.**

**Tel. +32-2-295.40.48 (office)**
**+ 32-2-654.09.44 (home)**
**Fax. +32-2-295.01.46**

UM

**^ y**

Annex II

**Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute for Advanced Materials** **(1AM) -** **Petten and** **Ispra**

Scope of Visiting Groups

The chief scope of Visiting Groups is to perform evaluations, taking into account
the objectives as set out for the 1992-1994 period (in Council documents, relevant
papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to
provide recommendations on the proper course to be followed by the Institute
visited in view of the stated objectives, and to apply in the evaluations, elements
of the methodology proposed for research returns and effectiveness.

The Visiting Group should:

perform an evaluation of all activities of the Institute against its stated
objectives, with particular emphasis on the impact the results of the work
has had on the user of this research, and the degree of collaboration
established with national research laboratories, as well as with industry;

assess the scientific quality of the work and the appropriateness of the work
and results for the Community;

judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the Community)
in comparison with similar national laboratories;

provide advice on the ongoing work on the basis of the evaluations and
assessments performed;

express recommendations for the future development of the Institute with
regard to the overall strategic plans for development of the JRC in the
medium-term and long-term.

Terms of Reference

1. To evaluate progress in performing work in accordance with the objectives
set out in Council Resolution 92/C118/03 and Council Decisions 92/273/EEC,
92/274/Euratom and 92/275/Euratom, documents addressed to the Board of
Governors, the JRC's Annual Workschedules, budget etc., and to review
progress in the implementation of the recommendations of the last
Evaluation Panel.

2. To ensure that the Institute has effective mechanisms to set its local scientific
objectives, monitor progress towards those objectives and evaluate outputs.

**t/é** **-2-**

**3.** **To review the various research activities to assess:**

**a)** **the scientific quality and productivity** **of** **the Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of funding and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management leadership within**
**the Institute.**

**5.** **To advise on whether the Institute has been making efficient, effective and**
**economical** **use** **of** **resources** **in** **carrying** **out** **its** **programmes** **and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the higher**
**education sector, industry and government departments in the Member**
**States,** **as well as outside** **the** **European Community.**

**7.** **To** **examine** **the relationship of the Institute's work in general, to the mission**
**of the Joint Research Centre and** **to** **its forward strategic planning.**

**8.** **To make recommendations and report to the Board of Governors of the**
**Joint Research Centre.**

**4V**

H a r w e l l, 1 5 . 1 1 . 9 3

**REPORT OF**

**VISITING GROUP FOR THE EVALUATION OF THE HIGH FLUX**

**REACTOR AT THE JOINT RESEARCH CENTRE'S INSTITUTE**

**FOR ADVANCED MATERIAL (PETTEN)**

**SUMMARY**

The HFR (Petten) is being managed and operated in a competent and professional manner.
The main research programmes using the HFR are likely to decline significantly. The JRC
management should produce a future strategy for the HFR in collaboration with the
sponsors/customers

**- 1**

_u%_

**^ §**

CONTENTS

Introduction

II Work in accordance with Objectives

**m** Efficiency

IV Overall Conclusions

V Recommendations

Appendix I Visiting Group Membership

Appendix II Visiting Group Terms of Reference

**2 -**

_$o_

**r**

**VISITING GROUP FOR THE EVALUATION OF THE HIGH FLUX**

**REACTOR AT THE JOINT RESEARCH CENTRE'S INSTITUTE**

**FOR ADVANCED MATERIAL (PETTEN)**

**INTRODUCTION**

At the invitation of the Board of Governors, JRC, the Visiting Group (for membership see
Appendix 1) attended the Institute for Advanced Materials - High Flux Reactor for a number
of days during June/July 1993. In total, some 15 man days were expended on the visit.

In that time, a wide range of technical and scientific topics were covered and therefore it
must be recognised that comments, conclusions and recommendations from the Visiting
Group are made as general supporting, constructive advice rather than specific criticisms of
the operations.

The specific scientific and technical objectives for the High Flux Reactor (HFR) are given
in Annex 1 to the Council Decision No 92/275/Euratom of 29 April 1992. The principal
objectives are:

1 The safe operation of the HFR in Petten. This comprises routine operation of the
facility for more than 250 days/year, fuel cycle management and safety and quality
management.

2 Efficient utilisation of the reactor in a wide scope of topical sectors: irradiation testing
of materials for fission reactors, as well as for future thermonuclear fusion reactors,
neutron applications to solid state physics and materials sciences investigations,
radioisotope production and related activities, neutron radiography as a nondestructive testing method, and treatment of certain types of cancer with neutrons
(BORON Neutron Capture Therapy) and related research.

The more general objectives are concurrent with the Joint Research Centre's (JRC)
mission and are highlighted in the scope for the Visiting Groups. The Visiting Group
should:

- perform an evaluation of all activities of the Unit against its stated objectives, with
particular emphasis on the impact the results of the work has had on the users of this
research, and the degree of collaboration established with national research
laboratories, as well as with industry;

- assess the scientific quality of the work and the appropriateness of the work and
results for the Community;

- judge all aspects of the efficiency and impact of the work, taking into account the
particular status of the JRC (its independence in the Community) in comparison with
similar national laboratories;

**- 3**

_$1_

provide advice on the ongoing work on the basis of the evaluations and assessments
performed;

express recommendations for the future development of the Unit with regard to the
overall strategic plans for development of the JRC in the medium-term and long-term.

This report will be organised following the order set out in the Terms of Reference
(see Appendix 2) with the Group's recommendations for each point summarised under
'Conclusions'.

**H** **WORK IN ACCORDANCE WITH OBJECTIVES**

     - The HFR is operated in a safe manner with due attention to safety and quality
management. It is operated with a high degree of availability for about 265 days per
year with very few unforeseen shutdowns.

     - The 'Work and Action Plan' for quality assurance is being implemented at an
adequate pace with all essential elements completed and regular reviews of the
ongoing activities which follow from the plan.

     - The fuel cycle, being dependent on the United States (US), is not currently closed due
to the ongoing review of the US' fuel policy.

     - Spent fuel elements are stored on the site which has adequate facilities for storage
(these can also be increased in increments) for the short and medium term.

     - The HFR uses High Enriched Uranium (HEU) fuel and has adequate suppliers for this
type of fuel in the medium-term. The technical and economical implications of an
eventual conversion to Low Enriched Uranium is being reviewed at present and so
far it appears to be technically feasible. The decision to change from HEU to LEU
remains a predominantly political one.

     - Particular attention was paid to the condition of the plant and its situation vis-a-vis
its operational safety. The HFR is found to conform to the requirements of the
Netherlands Kernfysische Dienst (KFD), the safety authority concerned. At the same
time, a programme of continuous upgrading is pursued to take advantage of
technological progress and to conform to the latest evolutions in good practice.

CONCLUSIONS

     - The HFR meets the first principal objective.

     - The HFR is a multi-purpose Oak Ridge Research-type (ORR-type) light water reactor
having a potential use for a wide range of sectors. It has been utilised in all the areas
mentioned in the second principal objective.

     - As far as the operational efforts are concerned, there has been an efficient utilisation
to the satisfaction of the 'customers'.

**- 4 -**

### **n**

The utilisation is dominated by 'customer-controlled' programmes with little overlap
and collaboration with JRC programmes, but adequately supported by the HFR
Division of the JRC.

Therefore, the responsibility for the scientific quality and rate of progress for the
programmes utilising the HFR lies mainly with the 'customers'.

EH EFFICIENCY

The Visiting Group was able to interview representatives from user organisations.

```
  NETHERLANDS

```

     - Netherlands' users provided most of their own design and manufacturing services for
their own experiments. They also assured day-to-day operation of the reactor under
contract and were therefore naturally disposed to comment favourably on efficiency.

     - It was felt that the Netherlands had received a fair share of the available positions up
to the present. However, there was now felt to be some over-capacity, particulary
with beam-tube utilisation where the facilities were no longer of international standard
and could be closed without serious loss of scientific impact, nevertheless, such work
is actively pursued elsewhere in the Netherlands and future requirements should be
carefully checked.

GERMANY

     - German project leaders from both KFA Julien and KFK Karlsruhe were present and
irradiation programmes concerning light water reactors, high temperature gas-cooled
reactors, fast reactors and thermonuclear fusion R&D were discussed.

     - All claimed that the service received from Petten staff was outstanding and the HFR
highly suitable in its technical characteristics, in its reliability and in the large number
of irradiation days per year. Petten engineers participated in project planning and
final reporting.

     - Unfortunately, due to reductions in government sponsorship, work on fast reactors
would be reduced to a small collaborative programme with Cadarache with perhaps
two core and one peripheral position in use in the HFR. This continuation was partly
due to the unique facility which allowed tests on fuel previously irradiated in Phénix.

     - A similar situation existed for HTR projects, where almost no work would be
continued. There might still be occasional irradiations on steel and special coatings,
but these would not be on a constant basis.

**5 -**

_**SH**_

_•_ Concerning light water reactor (LWR) fuel irradiations, the very flexible poolside
facility and the ability to employ it for testing power reactor fuels already burned up
in power plants would continue to provide some work. It was envisaged to use about
three poolside facility positions in the future.

 - In the absence of a dedicated 14 MeV neutron source for fusion materials R&D, the
HFR was still making valuable contributions in a variety of irradiation projects
concerned with 'first wall' materials, breeder materials and in the future some
component testing. Complementary work would also be performed in fast reactors.
Both types of reactor were needed for irradiation tests by the Fusion Programme at
the present time.

 - A dip in programme funding was expected in the near future, followed by a later
extension to new materials and perhaps components.

 - Overall, a reduction in present German activities by at least fifty per cent was
indicated.

COMMUNITY

- The Community Fusion Programme also planned to operate a modest programme of
materials irradiations on a regular basis to measure creep and other mechanical
properties in low activation structural materials. Irradiations on Tritium permeation
through breeder blanket component materials were also performed. The size of these
programmes was only limited by available funds.

- The JRC s Institute for Transuranium Elements at Karlsruhe hope to begin some
collaborative irradiations in a network with CEA and ECN beginning in 1995.

OTHER COUNTRIES

- Apart from Netherlands and Germany, little scientific work or scientists use the HFR.
As a Community facility there is a need to encourage wider use of HFR for research
and scientific training.

RADIOISOTOPES

- The primary use to which the reactor had been put was materials testing. However,
isotope production for medical and industrial purposes has been a 'by-product' of
HFR exploitation from the beginning. The Visiting Group received strong signals of
increasing interest in these possibilities from representatives of European isotope
manufacturers.

They expected to use up to thirty per cent of the reactor capacity, and realised that
in so doing, a commercial price would have to be paid for neutrons. This would

6

**5Y**

BNCT

imply an increase in the cost of the final product and such a change in policy could
only be introduced to the market at a reasonable rate.

However, with a decline in the materials irradiations' contribution to reactor costs,
isotope production could no longer be considered as a by-product of materials testing
and a normal tariff would have to be applied.

On the other hand, although the reactor was suitable technically and production
facilities were available, a more commercial attitude would have to be adopted by
both management and staff if industrial-scale production was attempted. Importance
should be given to ensuring maximum compatibility between materials testing
schedules and isotope production, taking into account operating schedules of other
European reactors producing medical isotopes. A considerably increased demand was
foreseen in the future.

Boron Neutron Capture Therapy (BNCT) was being developed at Petten and if
successful, could provide facilities to treat patients, initially with gliomas and later
perhaps with a number of other intractable tumours. This could provide an important
task for the reactor until other more convenient special purpose facilities could be
developed.

CONCLUSIONS

**•** The competence of the staff and the capability of performing difficult irradiation
studies using sophisticated techniques is recognised by the clientele.

The requirements for irradiations from the European nuclear power industry have
dwindled and a reorientation of strategy is badly needed.

An ability to respond more directly to commercial needs might be advantageous.

**- 7**

IV OVERALL CONCLUSIONS

To date, the operation of the HFR has been carried out in a satisfactory manner with:

          - high technological standards;

          - good operational performance;

          - high degree of customer satisfaction;

          - excellent safety record.

The JRC-HFR operation has been input-driven with no incentive to cut the cost of the
operation.

Utilisation to date has mainly focused on programmes originating in the sponsoring
countries both of which signal declining needs for the forthcoming programme period.
Only a small part of the utilisation has been on a Community basis.

The Fourth Framework Programme proposal, as already discussed in the Council,
does not indicate an increased Community need.

The HFR is facing a rapidly changing situation:

some German programmes have ceased, some are declining;
the Fourth Framework Programme proposal foresees further reduction in the
nuclear area (fission);
the needs of the future fusion programme have not been clarified at this stage;
the beam research at the HFR is no longer supported by the ECN and the
possibility of a complete closing down of such research must be foreseen;

the demand for radioisotopes is rising, however, the situation of NTD silicon
is uncertain and there seems to be limited additional European industrial need;
BNCT project is promising but is still at a speculative stage as a basis for a
potential customer programme.

All of the above means that the overall use will fall unless vigorous actions are taken
to bring in new customers.

The HFR operation has been demand-driven with management focus on operations,
safety and technological improvements.

There is a need for an output oriented strategy to ensure the continued efficient
utilisation of the HFR and its facilities.

The strategy should identify the future core-customer base and the optimal
organisation structure.

10

The core customer base could be:

- the fusion programme - well-established European programme;

- production of radioisotopes - catering to European-industry on a commercial
basis;

**- 8 -**

_**y**_
**f**

     - a fission programme concerning the fuel cycle and nuclear safety - transition
from national programmes to a Community-based programme.

     - BNCT - a European project with perspectives for a public service medical

programme.

11 The pursuit of these diverse needs and opportunities in a successful manner requires
proactive planning and constructive interactions between the HFR management and
the customers.

12 The organisation must be adapted to customer requirements with clearly defined
responsibilities for developing each of the core business areas.

13 An output-oriented approach will require sustained attention to cost-effectiveness.

14 If vigorous actions are taken now, a smooth transition from the present input-driven
programme (1992-1995), to a new output-oriented programme for the next
multiannual period could be achieved.

-9

**RECOMMENDATIONS**

**1** **As owner of the facility, the Commission should actively pursue the identification of**
**future requirements for the HFR.** **-.**

**2** **JRC management should generate a draft strategy (result - and output - oriented) for**
**the HFR in collaboration with the customers.**

**3** **Aim to achieve consensus from sponsors on the above strategy as a basis for a new**
**multiannual programme.**

**4** **On the basis of the above, decide optimal organisation structure and modus operandi**
**to implement strategy.**

**5** **Clearly identify responsibility for implementation of the strategy.**

**6** **The fuel cycle liabilities must be covered financially.**

**7** **The issue of responsibility for funding decommissioning must be resolved.**

**8** **In view of increasing commercial usage of the HFR reactor, there is a need to**
**develop a more autonomous financial and management structure to become output-**
**oriented.**

**9** **A vigorous and lateral approach needs to be taken to review all aspects of the cost**

**structure.**

**The Visiting Group are indebted to Professor E Hondros, and the JRC Directorate at Petten,**
**the ECN HFR staff who provided the detailed information and guided tours around HFR, and**
**the customers who were willing to come to Petten to be interviewed by the Visiting Group.**

**- 10-**

**y** **S**

**Appendix I**

**VISITING GROUP MEMBERSHIP**

Mr John Baxter (Chairman)
Director
Engineering Project Management
AEA Technology

Mr Lucien Gilles

Head of Testing Reactors Department

Centre d'Etudes de Saclay
Commissariat à l'Energie Atomique (CEA)

Dr Roberto Ramon Gonzalez Amado
Head of the Department of Engineering
Escuela Técnica Superior de Ingenieros y Industriales
Universidad Carlos HI

Dr Jorgen Kjems
Director, Research
Ris0 National Laboratory

Secretary

Mr Samuel J Lloyd

                  - 11

#### _i^_

_**éA**_

Appendix U

TERMS OF REFERENCE

1. To evaluate progress in performing work in accordance with the objectives set out in
Council Resolution 92/C118/03 and Council Decisions 92/273/EEC, 92/274/Euratom
and 92/275/Euratom, documents addressed to the Board of Governors, the JRC s
Annual Work Schedules, budget etc, and to review progress in the implementation
of the recommendations of the last Evaluation Panel.

2. To ensure that the Institute/Unit has effective mechanisms to set its local scientific

objectives, monitor progress towards those objectives and evaluate outputs.

3. To review the various research activities to assess:

a) the scientific quality and productivity of the Unit/Institute;

b) whether full advantage is being taken of scientific opportunities in the light of
funding and other constraints;

c) the relevance of the work with respect to Community needs, including the
relevance of work performed for external customers, when applicable.

4. To assess the effectiveness of the scientific management leadership within the
Institute/Unit.

5. To advise on whether the Institute/Unit has been making efficient, effective and
economical use of resources in carrying out its programmes and management
functions. Resources include manpower, money, services, facilities, data and
equipment.

6. To review the extent and effectiveness of the Institute's/Unit's external scientific

links, including relationships with research organisations, the higher education sector,
industry and government departments in the Member States, as well as outside the
European Community.

7. To examine the relationship of the Institute's/Unit's work in general, to the mission
of the Joint Research Centre and toits forward strategic planning.

8. To make recommendations and report to the Board of Governors of the Joint
Research Centre.

12

## _u_

_**éô**_

**Roskilde,** **UthJune** **1994**

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR**

**TRANSURANIUM ELEMENTS**

/y

**^JT**

**CONTENTS**

**I.** **Introduction**

**II.** **Activities at** **ITE**

**III.** **Specific** **Programmes**

1.1 Basic Safety Research in Nuclear Fuels

III.2 Safety Aspects of Fuel Operation and Handling

1.3 Actinide Determination and Recycling Activities

1.4 Characterisation of Waste Forms and of High Burn-Up Fuel

1.5 Actinide Research

**IV.** **Work for Commission Services**

**V.** **Work** **for** **Third Parties**

**VI.** **Exploratory Research**

**VII.** **Collaboration**

**VIII.** **Organisational Matters**

**IX.** **Output**

**X.** **Recommendations**

**Annex** **I:** Members of the Visiting Group

**Annex II:** Terms of Reference of the Visiting Group

_a_

14 June 1994

**REPORT OF THE GROUP FOR THE EVALUATION**

**OF THE JOINT RESEARCH CENTRE'S INSTITUTE FOR TRANSURANIUM**

**ELEMENTS** **(ITE)** **IN KARLSRUHE**

**INTRODUCTION**

A group of scientists experienced in various aspects of nuclear fuel and actinide
research and development (R & D) was invited by the Board of Governors of
the Joint Research Centre (JRC) to evaluate the work of the JRC's Institute for
Transuranium Elements (ITE) in Karlsruhe. Members (see Annex I) visited the
Institute on 21 and 22 February and assessed the Institute and its activities on
the basis of the Terms of References set out in Annex II.

The visit comprised a presentation of the Institute and its programmes by the
Director and presentations by senior scientists on the following activities:

nuclear fuel safety;
spent fuel characterization;
nuclear chemistry;
basic actinide research.

Discussions were held with staff responsible for organization, finance and
management of the institute, future scientific perspectives were examined and
a number of laboratories were visited.

Finally, members of the Group were able to meet representatives from clients
for whom third party contracts are being performed (e.g. Siemens and AlphaMedical (AKZO).

**ACTIVITIES AT ITE**

The Institute's programmes and projects during the 1990-1994 programme
period are distributed as follows:

  - Specific Programmes

Basic Safety Research on Nuclear Fuels
Safety Aspects of Fuel Operation and Handling
Actinide Determination and Recycling
Characterization of Waste Forms and High Burn-up Fuel
Actinide Research

_**&**_

Exploratory Research

Acoustic Aerosol Scavenging with Ultra and Infrasound
Study of Advanced Fuel Cycles
Thermal Imaging of Irradiated Fuel Rod Sections

Scientific - Technical Support to Community Policies

Development and Testing Methods; Execution of Safeguards Analysis;
Installation and Operation of On-Site Laboratories for Safeguards
Analyses

Work for Third Parties (major contracts)

Post Irradiation Examination of LWR fuel rods

Post Irradiation Examination PHEBUS

      - ' Interaction Fuel-Cladding

Preparation, Irradiation and PIE of Minor Actinide Containing Fuels
Fuel Analysis and Characterization
Preparation of Radionucleides for Therapeutical Application

**III** **SPECIFIC PROGRAMMES**

These areas comprise work performed under the Third Framework Programme
and they are part of the following JRC-programmes:

Reactor safety
Radioactive waste management
Actinides in nuclear fuel cycle safety

The specific programmes of the Institute as a whole depend on its expertise
and equipment in the field of transuranium elements, especially the actinides.
Although the specific programmes cover quite a large scientific and technical
area a common feature is the highly specialized hot cells sealed against escape
of alpha particle contamination and shielded against high energy gamma
radiation. These very expensive installations in combination with the
experienced staff give the Institute a unique position in European and
international nuclear research. This position is recognized worldwide and should
be maintained even if the interest in nuclear energy at present is declining in
Europe. This recommendation is based on the merit of a number of individual
research programmes which will be described shortly in the following.

_**1. Basic safety research in nuclear fuels**_ covers a number of projects mainly
concerned with fuels for light water reactors (LWR's). In parallel to experimental
studies a fuel performance code TRANSURANUS is developed and validated.
This work is important in the light of the many LWR's operating in Europe as
it ensures continuity in European research on both basic and applied problems

**<*•?**

with impact on safety and economy. The activities within this programme related
to a number of projects carried out for third parties, especially the post
irradiation examination (PIE) work. However, due to commercial restrictions the
research activities cannot fully benefit from the results from such PIE work. This
has a drawback and a solution should be sought when future contracts are
negotiated.

_**2. Safety aspects of fuel operation and handling**_ deal with investigations of
advanced fuels and with transport and dispersion of radioactive aerosol
particles in nuclear facilities. Individual projects within this programme have
important safety aspects for different parts of the fuel cycle. However, the
overall objective of'this programme could be defined more stringently.

_**3. Actinide Determination and recycling activities**_ are mainly concerned with
the transmutation of long lived nuclear waste constituents (minor actinides) into
shorter-lived fission products. This programme is an excellent example of the
special expertise of the Institute being applied to a problem of high relevance
to society in dealing with waste from past and present nuclear facilities. It is
also within this area that the work for third parties has been economically most
successful.

_**4. Characterization of waste forms and of high burn-up fuel**_ covers efforts
to model the consequences of water intrusion into an underground fuel deposit.
As is the case for programme III 3 this programme aims at reducing the risk of
storing nuclear waste. It is a good example of the uniqueness of the Institute
in combining specialized experimental work with high level theoretical work
including computer modelling.

5. _**Actinide research**_ covers basic research of the physical and chemical
properties of the actinide group of elements. Within this area the Institute has
an internationally acknowledged expertise both theoretically and experimentally.
These in-depth studies of actinides are important to extend our knowledge of
this important group of materials but the research is also necessary to
understand a number of technological problems related to the fuel cycle. This
research on actinides should be continued but its uniqueness in the European
framework could be more emphasized. One possibility is to promote the
facilities and expertise as a European laboratory of excellence open for
scientists from the member states in parallel to laboratories and installations
participating in EU's Large Installation Programme (LIP). It is realized that such
a status as a user-oriented facility will require extra resources, especially new
staff. However, in all cases a consideration of the staff situation in this area is
urgent in order to ensure continuity as senior staff is now near retirement.

**IV** **WORK FOR COMMISSION SERVICES**

The safeguarding of nuclear materials is an important task of the Commission
(DG I and DG XVII) and of the International Atomic Energy Agency (IAEA). The

_**-h**_

Institute is able to contribute effectively to this important task. The Institute's
chemical expertise is exploited in specialized chemical analyses of samples
taken from nuclear installations by safeguards inspectors. An automated in-cell
analytical machine has been developed at ITE to improve accuracy and
convenience and has been widely adopted elsewhere.

As a further development of this service, the Institute is providing analytical onsite laboratories at Sellafield and Cap la Hague which will perform the bulk of
these analysis.

In the area of nuclear safeguarding in Europe the Group must point to the
essential part now being played by the Institute, Its success being attributed to
its expertise in a number of disciplines.

**WORK FOR THIRD PARTIES**

The group applauded the Institute' success in winning contracts - some of them
of very long duration. From 1990 up to the end of January 1994 orders have
been placed for work worth about 18 MECU. In total 23 contracts have been
received and it is noticed that two large contracts received in 1990 and 1991
amount to 85% of the total contract volume. The remaining 21 contracts are all
below 0.5 MECU with an «average about 0.1 MECU. In the period 1992 - 1994
(per 15/2) the Institute has received 18 contracts amounting to about 2 MECU.
About 60% of this volume (9 contracts) falls within the area of nuclear
chemistry.

The discussion of the relationships behind the Institute and its clients included
two interviews summarized in the following:

A representative from Siemens (Dr. Manzel) was appreciative of the
Institute's activities under contract on high burn-up fuels for LWR's.
Siemens' hot cells had been closed some years previously and the
transfer of work to ITE has proved to be very satisfactory. Client/contractor
relations were excellent and at present it was expected that the
association would continue in future. The Institute's staff, equipment and
service provided were felt to be excellent in quality and administrative
procedures relatively light. Scientific collaboration with the staff of the
Institute worked well in certain areas, whereas other fields, for example
fuel performance code activities were excluded for proprietary reasons.

A representative of Alpha Medical (Dr. Geerlings) was also appreciative
of the Institute's work on a contract concerning the separation of certain
radioactive actinide isotopes in the production of a radiopharmaceutical
agent for the treatment of certain cancers. The Institute's skills and
installations were being used to develop suitable separation chemistry and
Alpha Medical hoped to launch clinical trials in some months in
collaboration with an institute in New York.

**?l**

The group discussed in general the Institute's work for third parties and noted
that the contracts received fitted well with the core programmes, thus in many
cases a synergetic effect can be expected. The Group was also pleased to see
the facilities of the Institute being applied on projects of commercial interest. It
was realized that in such cases full openness of the work cannot be expected,
but it was emphasized that results from third party work, whenever possible,
should be made available for the Institute's own research possibly with certain
restrictions on publication.

**VI** **EXPLORATORY RESEARCH**

The Visiting Group welcomes the 5 - 6% exploratory research carried out at the
Institute. This was seen both as a means of exploring possible new areas of
activity for the Institute and of trying out ideas for later incorporation into core
activities.

However, attention is drawn to acoustic agglomeration techniques which have
been subject for some years of repeated studies covering various possible
methods or applications. When a technique or discovery is proven by an
exploratory activity, it should either be exploited by commercial development or,
if of a suitable nature, incorporated into the mainstream of the Institute's
scientific activities. This vyill then release funds for new exploratory research
which should not be pursued further than proof of the principle.

The group discussed specifically the project "Evaluation of Potentialities of
Alternative Fuel Cycles" and it was felt that this activity at present should be
limited to theoretical work and should not include planning of irradiation
experiments. The reason for this recommendation by the group is the lack of
European accord on the problems proposed to be investigated.

**VII** **COLLABORATION**

The group acknowledged the widespread international collaboration in
practically all research areas of the Institute. The group was, however, surprised
to learn about the very limited exchange of scientists taking place for longer
periods between the Institute and its collaborators.

**VIII** **ORGANIZATIONAL MATTERS**

1. The group inquired about the Institute's organization vis a vis the
administrative structure of the JRC. Of special interest to the Group was
the technical and administrative responsibilities on the different levels at
the Institute. The Director explained how the system worked which was
more or less according to tradition as detailed written procedures had not
been worked out. The group did not pursue this point but suggests it for

_**>**_

future consideration.

In order to evaluate the activities of the Institute against its stated
objectives the Group asked for a description of such objectives. Such a
description was given verbally by the Director but written documentation
was not available. It is, however, the general impression that overall
objectives are adhered to in the various programmes. However, the lack
of well-defined written objectives makes it impossible for the group in more
detail to evaluate activities against objectives.

The project/programme management system at the Institute appears to
operate effectively and to make good use of the time available on the
various installations. Also the general administration at the Institute
appears to be satisfactory. In general, the Group noted that the
organization of the Institute in units with almost a constant number of staff
over the years may lead to a too rigid planning and some lack of freedom
in the formulation of new projects.

The staff situation was discussed and the Group expressed its concern at
the age structure of the institute. Much of the Institute's experience and
expertise is concentrated in staff in charge of units and laboratories who
will be retiring in the relatively near future. Future planning would have to
take into account tfre need to transfer knowledge and know-how to
successors who would have to be chosen well in advance. Recruitment
to the JRC presents many difficulties, both in the selection system used
in the Commission (difficulties in the specification of requirements) and in
the length of the process. A positive trend has been the increase in post
doctoral fellows which has increased from 2 in 1991 to 17 in 1994, mainly
sponsored by the Mobility Programme. This was considered highly
satisfactory as young scientists now can qualify and hopefully present
themselves as candidates for future positions at the Institute.

**IX** **OUTPUT**

The number of articles published in scientific journals increased from 35 in 1990
to 40 in 1993, together with a similar trend in contributions to conferences which
increased from 72 to 106 respectively. This is considered to be a good output
of high scientific quality for such an institute considering that much of its product
is issued in the form of data, analytical results, etc.

**X RECOMMENDATIONS**

1. The application of present and future knowledge and expertise existing at
ITE to the problems of the safe management of nuclear energy,
particularly in dealing with the sensitive area of nuclear waste is an
important objective. This mission will remain of high significance in the

**?3**

foreseeable future for Union Member States, whatever the outcome of
debates on the use of nuclear fission energy and must remain the focus
of development at the Institute. In such development nuclear chemistry
appears to be an important growth area contributing to a large number of
research projects.

2. In view of the continuing need for research and information on the actinide
elements and their behaviour, together with the very restricted number of
institutes where such work can be carried out, the Group strongly
recommends that efforts be made to continue and develop the ITE core
activities with emphasis on the basic understanding of actinide chemistry
and physics. To accomplish this task effectively it is essential that the level
of staff excellence is maintained. It is suggested that facilities and staff
within this area are expanded in order to establish a "user facility" for
European scientists similar to those within the "Large Installation
Programme".

3. Work in support of other DGs is clearly an essential role of the JRC and
the Institute, as its specialists can provide important help in a number of
areas. The Group welcomes the use made of the ITE by DG I (IAEA) and
DG XVII (Euratom) for the analysis (both on-site and in the Institute) of
samples taken by safeguards' inspectors from nuclear plant and facilities.
The contribution by the Institute is very considerable and must continue.

4. Making available services and facilities to the outside world under
contracts is an important aspect of the Institute's activities and is to be
strongly encouraged. At present this activity is on a satisfactory level but
a certain increase should be planned for taking the following points into
consideration: (i) The work for third parties should as far as possible
support the Institute's own research and the results should be made open
at least to the scientists at the Institute for their own use. (ii) In the
promotion of work for third parties it must be assured that the staff
distribution at the Institute is so flexible that successful units can get the
necessary support, (iii) The amount of contract work should not be allowed
to increase to a level where core activities might be jeopardized.

5. Attention should be focused on exploratory research to make certain that
it really is "exploratory" in nature rather than repetitive and where possible,
points to potential new developments in the Institute's core activities in
science and technology. New work on alternate fuel cycles is not
encouraged as it does not appear to fulfil the above requirements.

6. The project/programme management system appears to operate efficiently
and to make good use of the time available on the various installations.
However, the manpower distribution into units of almost constant size over
the years appears to be too rigid both in establishing new research
priorities and in promoting work for third parties.

_**V**_

**7.** **Serious consideration has to be given to ensuring that staff replacements**
**will be of exceptional quality if the Institute is to maintain its position in the**
**scientific world. The present system does not appear to** **be** **either**
**adaptable enough nor rapid enough to meet** **that** **end and the exact**
**specification of** **requirements** **for scientific posts must lie with the Institute**
**rather than centrally.**

**8.** **If resources can be found, the fruits of the work at the Institute could be**
**best disseminated by further increasing the numbers of post graduate and**
**post doctoral fellows in the Institute. At present levels of scientific A staff,**
**some** **15** **Postgraduates and 25 Post Doctoral fellows might be foreseen.**
**In this connection the exchange for longer periods of scientists is**
**encouraged to establish more** **firm** **and vivid contacts between the Institute**
**and the universities and national laboratories with which it collaborates.**

**8**

**)**

**Annex** **I**

**Visiting Group to the Joint Research Centre's**
**Institute for Transuranium Elements (ITE) - Karlsruhe**

**Members**

**Dr.GuyBAUDIN,**
**Head of Department,**
**Commissariat à** **l'Energie**
**Atomique (CEA),**
**31 rue de la Fédération,**
**F-75752** **Paris.**

**Dr. Niels** **HANSEN** **(Chairman),**
**Head of Department,**
**Materials Department,**
**Ris0 National Laboratory,**
**Frederiksborgvej 399,**
**P.O.** **Box** **49,**
**DK-4000 Roskilde.**

**Prof.** **Dr. Georg** **KALVIUS,**
**Head** **of** **Institute,**
**Physics** **E15,**
**Technische** **Universitat** **Munchen,**
**James-Franck Str.,**
**D-85747 Garching.**

**Secretary**

**Mr. Samuel J.** **LLOYD,**
**Avenue Reine Elisabeth 8,**
**B-1330Rixensart.**

**Tel. + 33-1-69.08.91.56/69.08.91.60**
**Fax. + 33-1-69.08.91.47**

**Tel.** **+ 45-46-77.57.01**
**Fax. + 45-42-35.11.73**

**Tel. + 49-89-32.09.25.01.02**
**Fax. + 49-89-320.67.80**

**Tel. + 32-2-295.40.48 (office)**
**+ 32-2-654.09.44 (home)**
**Fax. +32-2-295.01.46**

###### _V_

_**?î**_

**Annex** **II**

**Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute for Transuranium Elements (ITE) - Karlsruhe**

**Scope of Visiting Groups**

**The chief scope of Visiting Groups is to perform evaluations, taking into account**
**the objectives as set out for the 1992-1994 period (in Council documents, relevant**
**papers for the Board of Governors, JRC Annual** **Workschedules,** **budget, etc.), to**
**provide recommendations on the proper course to be followed by the Institute**
**visited in view of the stated objectives, and to apply in the evaluations, elements**
**of the methodology proposed for research returns and effectiveness.**

**The Visiting Group should:**

**perform an evaluation of all activities of the Institute against its stated**
**objectives, with particular emphasis on the impact the results of the work**
**has had on the user of this research, and the degree of collaboration**
**established with national research laboratories, as well as with industry;**

**assess the** **scientific** **quality of the work and the appropriateness of the work**
**and results for the Community;**

**judge all aspects of the efficiency and impact of the work, taking into**
**account the particular status of the JRC (its independence in the Community)**
**in comparison with similar national laboratories;**

**provide advice on the ongoing work on the basis of the evaluations and**
**assessments performed;**

**express recommendations for the future development of the Institute with**
**regard to the overall strategic plans for development of the JRC in the**
**medium-term and long-term.**

**Terms of Reference**

**To evaluate progress in performing work in accordance with the objectives**
**set out in Council Resolution** **92/C118/03** **and Council Decisions 92/273/EEC,**
**92/274/Euratom and 92/275/Euratom, documents addressed to the Board of**
**Governors, the** **JRC's** **Annual Workschedules, budget** **etc.,** **and to review**
**progress in the implementation of the recommendations of the last**
**Evaluation Panel.**

**2.** **To ensure that the Institute has effective mechanisms to set its local scientific**
**objectives, monitor progress towards those objectives and evaluate outputs.**

**?«**

**-2**

**3.** **To review the various research activities to assess:**

**a)** **the** **scientific** **quality and productivity of the Institute;**

**b)** **whether** **full** **advantage is being taken of scientific opportunities in the**
**light of funding and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management leadership within**
**the Institute.**

**5.** **To advise on whether the Institute has been making efficient, effective and**
**economical** **use of resources in carrying out its programmes and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the higher**
**education sector, industry and government departments in the Member**
**States,** **as** **well** **as** **outside the European Community.**

**7.** **To examine the relationship of the Institute's work in general,** **to** **the mission**
**of the Joint Research Centre and to** **its** **forward strategic planning.**

**8.** **To make recommendations and report** **to** **the Board of Governors of the**
**Joint Research Centre.**

**?s**

London, 2nd December 1993

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR SYSTEMS ENGINEERING**

**AND INFORMATICS**

tfo

**s**

**CONTENTS**

**I.** **Introduction**

**II.** **General Observations**

**III.** **Scientific** **and Technical Activities**

**111.1** **Block** **1** **- Environment Hazards, Industrial Environment**

**111.2** **Block 2 - Renewable Energies**

**111.3** **Block 3 - Nuclear Energy**

**111.4** **Block 4 - Information Technologies**

**IV.** **Management of Research**

**IV.1** **Mission**

**IV.2** **Transfer**

**IV.3** **Quality**

**IV.4** **Areas of Concern**

**V.** **Executive Summary**

**VI.** **Recommendations**

**VI.I** **Infrastructure**

**VI.2** **Management**

**VI.3** **Informatics**

**VI.4** **Competition**

**Annex** **I:** Members of the Visiting Group

**Annex II:** Terms of Reference of the Visiting Group

_% l_

**S3>**

**Report of the Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute** **for Systems Engineering and Informatics,** **Ispra,** **Italy**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), members of the Visiting Group (see Annex I) spent the week of 1115 October 1993 at the Institute for Systems Engineering and Informatics
(ISEI) at Ispra reviewing the Institute's work and visiting its laboratories.

The objectives for the Institute are to be found in the Technical Annex to
the JRC Programme Proposal 1992-1994 (COM (91)281 final). These are:

"to be the European centre of excellence in the field of 'complex systems'
engineering, focussing on the evaluation and development of methods
addressing the safety, standards and performance of systems which
involve industrial, social and environmental issues on a pan-European
scale.

In particular, to take the broad 'systems approach' to the understanding
of the human, societal and environmental implications of scientific,
technological and industrial developments in order to provide technical
advice for the balanced social and industrial development of European
society.

The Institute for Systems Engineering and Informatics will shortly be
launching new initiatives in the areas of Environmental Accident
Management, Safety at Work and later, Safety Critical Computer Systems
and a contribution to the proposed Centre for Earth Observation. This
work will build on the Institute s existing strengths in systems engineering
and informatics". (These initiatives have since been implemented.)

These general objectives are concurrent with the JRC's mission and are
highlighted in the paper defining the scope of the Visiting Groups. As set
out in the terms of reference (see Annex II), the Visiting Group should :

    - evaluate all activities of the Institute against its stated objectives,
with particular emphasis on the impact the results of the work has
had on the users of this research, and the degree of collaboration
established with national research organisations, as well as with
industry;

     - assess the scientific quality of the work and the appropriateness of
the work and results for the Community;

     - judge all aspects of the efficiency and impact of the work, taking
into account the particular status of the JRC (its independence in the
Community) in comparison with similar national bodies;

     - provide advice on the ongoing work on the basis of the evaluations
and assessments performed;

     - express recommendations for the future development of the
Institute with regard to the overall strategic plans for development
of the JRC in the medium-term and long-term.

**VI**

**-2**

**II.** **General Observations**

**For the purposes of this report, scientific and technical activities of the**
**Institute are organised into four blocks following the scheme adopted for**
**their presentation.**

**These are:**

**1.** **Environment Hazards, Industrial Environment;**
**2.** **Renewable Energies;**
**3.** **Nuclear Energy;**
**4.** **Information Technologies.**

**The Visiting Group recognises the five-part purpose of the** **Institute:**

**to serve the** **European Community** **by performing work** **for**
**Framework Programme specific activities;**

**to render support to other Directorates General;**

**to serve the JRC by providing informatics services and advice to**
**Institutes and Headquarters;**

**to perform work under contract for third parties, making available**
**expertise and facilities to European industry;**

**to carry out exploratory research in order to maintain scientific**
**vitality in the Institute's staff and to develop tools for the future.**

**Members were impressed by the diligent dispatch of routine, albeit**
**important, tasks by the Institute (for implementing the MARS project -**
**Major Accident** **Reporting** **System - for example)** **but** **also** **noted**
**considerable creative** **wort** **(e.g. development of new measurement**
**techniques for European Solar Testing Installation (ESTI) etc., and in**
**techniques for non-intrusive surveillance).**

**Within the constraints of time, the Group applied three tests in reviewing**
**each scientific and technical activity:**

**•** _**mission**_ **- is the mission appropriate? Is it adequately described? Is it**
**sound?** **Is** **it followed ?**

**•** _**transfer**_ **- how well do relevant facts, skills and ideas travel upstream,**
**downstream and between parallel bodies both within and without**
**the JRC** **?** **What are the products ?**

**•** _**quality**_ **- is the work rigorous, fresh and relevant ? Is the value** **of**
**results commensurate with resources consumed ? Are management**
**methods appropriate ?**

**3-** **&f**

**111.** **Scientific and Technical Activities**

**111.1** **Block 1** - Environment Hazards, Industrial-Environment

The following activities are encompassed:

      - industrial hazards (plant safety and reliability, safety critical
computer systems);

      - aircraft incident reporting system;

      - working environment;

      - accident prevention;

      - reliability modelling of structures;

      - industrial hazards (environmental planning);

      - support to DG XI - Directorate General for Environment, Nuclear
Safety and Civil Protection        - (directives on major hazards,
biotechnology, environmental impact assessment, civil protection);

      - optical and holographic technologies applied to diagnostics and
information processing;

      - .Centre for Earth Observation (CEO).

_Comments_

_•_ The extension of techniques developed for the safe operation of
nuclear plants (where this has been the norm), to complex
installations in other industrialised activities was welcomed.

      - Members of the group were pleased to note the efforts to take
human factors into account when considering industrial safety.
Psychology-sociology disciplines were used rigorously in association
with physical science and engineering.

In projects such as the FORMENTOR project (EUREKA project No. 19)
and STARS (Software Tools for the Analysis of Reliability and Safety),
the practical implications for the safe and economic operation of
process plant were appreciated. The close interest shown by industry
was noted (installation of a system by British Petroleum). Attention
to the modification of plants over time would show benefits over the
whole life of complex installations.

      - Concerning the sub-project investigating stress among air traffic
controllers, it was felt that such information might already be
obtainable from Eurocontrol or the U.S. authorities, and that the
study might only reinvent existing techniques. It was, however,
sound to take this highly stressed working environment as a model
for human stress in other working situations.

      - The work on safety in complex systems and the interaction of human
beings with such systems was considered to be useful in stimulating
work outside the Institute and in supporting EC policy in the field.
The concepts could well be extended to non-lethal systems such as
funds transfer, trading and telecommunication applications.

      - The studies on optical and holographic applications to non-intrusive
surveillance of surfaces and solids were felt to be of high quality with
valuable future applications still to be explored.

_**té**_

**- 4 -**

**Conceming** **the Centre for Earth Observation (CEO), the Group saw**
**an important role for the JRC in converting the volume of currently**
**unintelligible satellite data into user-friendly input for diverse**
**applications.** **In analogy, thirsty people need help, if they are to**
**drink from a fire hydrant. This role seems particularly appropriate to**
**the JRC, which will be seen as neutral and unbiased by eventual users**
**of** **the data.**

**III.2** **Block 2 - Renewable Energies**

**This block comprises:**

**photovoltaic systems;**
**transfer of solar technology;**
**thin film photovoltaic devices in buildings;**
**solar energy and saving;**
**solar regional programmes.**

_**Comments**_

**The Visiting Group recognised the relevance of this work to**
**developing the photovoltaic industry in Europe. It was impressed by**
**the urgency created by close service contacts with** **numerous**
**manufacturers,** **several** **with** **rapidly** **changing** **technologies.**
**Response to requests for service and mastery of the techniques**
**involved were impressive;**

**knowledge of industrial costs and understanding state-of-the-art**
**technology were essential to the success of these activities and for**
**fruitful relations with industry;**

**the expected accreditation of the ESTI laboratory to** **ISO** **9000/CEN**
**29000 standards and the formal status of Calibration and Type**
**Approval** **certificates** **for this leading-edge activity were welcomed;**

**the Systems Analysis and Monitoring of Projects (ESAS) activity was**
**found to provide important support to a range of DG XVII (Energy)**
**policy actions and demonstrations.** **However, Members of the**
**Visiting** **Group were not satisfied that the whole economic envelope**
**was being addressed in the work on energy saving.** **Full,** **whole life**
**economic consequences should be included in decision support**
**systems if they are to be respected in the commercial world;**

**norms and standards which reflect leading-edge technology were**
**clearly crucial to the efficient working of the European market in**
**photovoltaic and energy saving products and processes.**

**III.3** **Block 3 - Nuclear Energy**

**This block comprises:**

**fission safety;**
**safeguards and fissile materials management;**

**nuclear safeguards support including collaboration with the**
**IAEA;**
**instrument and data evaluation for nuclear safeguards, seals**
**and tags;**

**«7-**

**expert** **systems for energy auditing, reliability and risk evaluation;**
**support to the harmonisation of nuclear safety for thermonuclear**
**fusion;**
**operational** **safety,** **accident analysis and environmental studies;**
**neural networks for navigation of a teleoperated vehicle.**

_**Comments**_

**Work on nuclear** **safeguards** **and safety of nuclear plants is a**
**traditional JRC core function, operating in a highly disciplined**
**environment.** **Research & Development is needed to meet new**
**requirements** **and** **to** **improve** **established** **methods.** **The**
**development** **of** **tamper-proof** **sealing** **bolts,** **upgrading** **of**
**instruments** **and the** **introduction** **of** **advanced** **management**
**techniques for fissile materials were noteworthy examples.**

**In the interests of non-proliferation of military nuclear materials and**
**their possible conversion to civil applications, it was felt that the JRC**
**should also be allowed to contribute to aspects of nuclear safeguards**
**relating to decommissioned military material.**

**The further development of Probabilistic Safety Assessment (PSA)**
**techniques in monitoring and controlling safety throughout the**
**lifetime of nuclear plants was of clear importance.**

**The** **Group** **was** **concerned** **that** **current** **informal** **project**
**management would prove inadequate where a number of JRC**
**Institutes located on several geographically separated sites are**
**involved in connected programmes.** **The work** **on nuclear safeguards**
**is a case in point.**

**Work on safety standards and engineering codes of practice for**
**fusion engineers was clearly important in supporting the practical**
**implementation of a thermonuclear fusion reactor.**

**The Community Fusion programme is fully integrated and tightly**
**managed (led by DG** **XII),** **and the JRC's contribution of some twenty**
**years standing is a small but important group of activities. The role**
**of** **ISEI** **as a test and validation centre for robotic blanket handling**
**devices, was felt to be appropriate to the Institute and should be**
**preserved. The practical experience gained will no doubt influence**
**design of** **fusion** **blanket elements in the ITER project.**

**Block 4 - Information Technologies**

**The Institute is required to provide informatics services to the JRC and to**
**other Commission Directorates General, while its budget for exploratory**
**research is minimal.**

**The in-house service function is in transition from a monopolistic** **main-**
**frame-based computing centre to a distributed user-friendly network,**
**with the Institute providing the communications backbone and some**
**advice to the user Institutes.**

**•** **The Group became aware that some difficulties are arising in the**
**necessary balance between freedom of choice by user Institutes and**
**network integrity.**

**-6**

**•** **Some network disciplines had to be universally recognised, in the**
**interests of** **inter-operability** **and network security; the latter**
**appeared to receive inadequate attention at present.** **The Unit**
**should be responsible for:**

**network architecture at communication and data formatting**
**levels;**

**evaluation of new** **tools,** **so** **as** **to advise researchers throughout**
**the** **JRC** **on value for money and best practice;**
**Information Technology auditing and transfer of good practice**
**throughout the** **JRC** **by training and other means.**

**However, Institutes should be encouraged to acquire freely informatics**
**and advanced computing tools to meet their specific needs, drawing on**
**experience elsewhere in their scientific fields where appropriate. The role**
**of** **ISEI** **should be to encourage best practice, and maintain essential**
**network** **discipline.**

**•** **The corporate informatics service has an important role to play in the**
**long** **overdue reform of the infrastructural and administrative**
**services** **(ADIN)** **so as to rationalise management and financial**
**systems, and to make better use of available manpower.**

**•** **Several examples of excellence and innovation were observed, but**
**more emphasis should be placed on fail-safe systems.**

**•**
**Use should be encouraged of informatics packages and advanced**
**computing tools developed outside the Institute.**

**IV.** **Management of Research**

**IV. 1** **Mission**

**•** **Objectives were found to be appropriate to the JRC and ISEI;**
**however, they are rarely defined in the detail necessary for effective**
**project control, (Exceptions are DG XVII and** **IAEA** **projects and DG**
**XII fusion projects). Milestones are lacking and there appears to be**
**little formal project management, except in some individual cases,**
**often in response to external pressure.**

**•** **A clear need is evident for a JRC-defined project management**
**system for pan-JRC programmes such as nuclear safeguards. Those**
**projects where more than one Institute is required to contribute**
**might better carry "JRC status". While one Institute might well be**
**assigned the leadership role, it should be exercised on behalf of the**
**JRC,** **with authority to manage project resources** **within** **defined**
**limits.**

**IV.2** **Transfer**

**•** **Problems were perceived at working level in promoting best practice**
**across Institute boundaries. However, transfer both inward and**
**outward with the rest of the JRC and the outside world was**
**improving.**

**-7**

**In certain areas, the Institute remained introverted and more use**
**should be made of external experience and tools, rather than solving**
**problems from scratch.**

**In** **areas such as FORMENTOR, the Fusion activities and ESTI, among**
**others,** **great benefit was derived from contacts with industry, and**
**several advanced techniques likely to be of value to industry existed**
**in the Institute.**

**Obviously, in the case of third party contract research, transfer was**
**inherent.**

**IV.3** **Quality**

**While much of the technical work is routine, such activities are**
**performed dutifully and to a good standard and level of expertise.**

**Several areas of excellence demonstrate creative scientific and**
**engineering work, for example:**

**ESTI;**
**holographies** **and optical systems;**
**robotics;**
**failure analysis (in fusion and elsewhere).**

**Much of ISEI does not employ rigorous project management, with**
**clear** **milestones,time** **schedules and control . Experience elsewhere**
**shows that pressure for results generates peer pressure which allows**
**little latitude to freeloaders.**

**IV.4** **Areas of Concern**

**The Visiting Group identified a number of areas where improvements**
**would lead to the better management and operation of ISEI. In some**
**cases,** **problems arise from elsewhere in the JRC, and not infrequently**
**from the requirements of the European Commission itself.**

**IV.4.1** **ADIN: Administrative services at Ispra do not support management of the**
**Institute effectively or economically.**

**•** **Overheads are excessive by external criteria; over 25% of manpower**
**on the** **Ispra** **site is employed in administrative and infrastructural**
**activities,** **including** **22 managers and administrators in the Institute**
**itself.**

**•** **Financial and budget information is complex and confusing.**

**•** **Overhead allocation and pricing rules are rigid and uncommercial.**

**•** **Personnel** **management** **practices are** **inflexible** **and** **slow** **in**
**recruitment, appointment, promotion, transfer and dismissal of**
**staff.**

**S3**

**-8**
_**%**_

**IV.4.2 Coordination within the JRC gave grounds for concern, as Institutes have**
**accepted encouragement to act with more self-reliance.** **Where**
**coordination is imposed by external** **clients,** **cooperation seems to be**
**effective:**

**•** **In the safeguards project, management is only effective because the**
**previous programme manager is still actively engaged. However,**
**Fusion activities are well coordinated among the Institute for**
**Advanced Materials** **(1AM)** **and the Institute for Safety Technology**
**(1ST) as well as** **ISEI.**

**•** **ISEI has problems at working level with the Institute for Remote**
**Sensing Applications and the Environment Institute.**

**•** **where lead Institutes are identified, project management is**
**hampered by the strict separation of Institute funds and by virement**
**rules.**

**IV.4.3 Mechanisms for exerting scientific leadership were not impressive. There**
**is a clear opportunity to foster improved transfer of scientific knowledge**
**and the development of professional scientific and management skills.**
**This could be encouraged by weekly seminars, on scientific subjects or**
**organisation given by local staff or visitors. The Group was pleased to hear**
**that seminars on project management had recently been held at ISEI.**

**•** **The local Scientific Committee is a staff representation body**
**inappropriate for scientific leadership.**

**•** **There is patchy transfer of best practice, both into and out of the**
**Institute.**

**•** **An undue proportion of students and grantholders in some ISEI**
**projects, hampers continuity.**

**•** **Central library facilities at Ispra are inadequate with**

**fragmentation among many locations;**
**poor reference section;**
**inadequate information service.**

**IV.4.4 Ability to Compete: while ISEI clearly has scientific skills and** **knowledge** **of**
**considerable commercial value, its ability to compete for research business**
**is hampered.**

**•** **Current CEC rules prevent** **JRC** **Institutes from receiving any EC funds**
**other than those specifically allocated to the JRC in the Framework**
**Programme.** **This effectively prevents ISEI from collaborating in**
**many European scientific programmes.**

**•** **Pricing rules are uncommercial: the value of intellectual property is**
**largely ignored, and discounted exploratory work** **is** **precluded.**

**•** **Rigid staff and virement rules impair ability to man and fund project**
**teams for third party work.**

**•** **There** **is** **an overwhelming concern for politics and procedures within**
**the** **JRC.**

**- 9 -**

**V.** **Executive Summary**

The Visiting Group was satisfied that the work undertaken by the Institute
is appropriate to its functions, and generally of high quality. However
objectives, while understood, were only rarely statedin a form that would
allow effective project management - except where outside bodies had so
stipulated. For those projects that required cooperation among Institutes,
sometimes only informal means of coordination were apparent. It was
felt that clear objectives and efficient project management would
stimulate performance, generating peer pressure on individuals and
improving value for money.

The Institute is clearly hampered in its work by the procedures and
requirements stipulated by the European Commission. ISEI is not well
served by the administrative infrastructure, and financial information is
too complex for easy comprehension. Constraints on staff appointments,
virement of funds between budget lines, and pricing, undoubtedly impair
effective bidding for competitive business - ana indeed operational
control generally. The bar to receiving EC funds, other than those
particularly allocated, prevents the Institute from participating in many
Community projects to which it might well bring valuable skills.
Paradoxically there is no such ban on receiving fees from third parties
outside the European Community.

The informatics work of the Institute is not currently supported by
research funds. However, the Visiting Group believes that ISEI has a
valuable role to play in maintaining essential discipline in the JRC's
communications network, and in promoting best practice in advanced
computing and informatics techniques throughout the JRC. Some
research funds should be allocated to enable these tasks to be carried out
at the state of the art.

VI. Recommendations

VI.1 Infrastructure

Information and reports to management should be reformed to
meet operational needs. Fixed and variable costs for each project
should, in particular, be regularly reported. Virement rules should
be relaxed.

The Director should have greater flexibility in selecting, appointing,
promoting and removing scientific staff.

VI.2 Management

Systematic project control should be applied throughout the
institute's work. Objectives should be stated in clear terms, and
milestones set to mark progress, defining cost and time parameters.

For projects involving more than one Institute, the Director General
of the JRC should nominate a Project Manager within one engaged
Institute, and assign appropriate powers to adjust personnel
assignments and funds.

**<h**

_°ll_ -.10

**•** **Training in project management skills should be regularly provided,**
**and be extended to all scientific staff.**

**•** **The Director should set in place mechanisms for stimulating scientific**
**excellence,** **and disseminating best practice.**

**•** **Connections with** **other** **scientists working in parallel fields should be**
**further stimulated.**

**•** **Involvement of industry in the Institute's work should be extended.**
**Regular** **relationships - perhaps at the JRC level - with senior**
**industrial scientists would help to form objectives, develop**
**management skills and facilitate technology transfer.**

**VI.3** **Informatics**

**•** **ISEI should be assigned authority for system architecture in JRC**
**communication networks.**

**•** **ISEI** **should be encouraged to advise other JRC Institutes on "best**
**practice" in advanced computing and informatics.**

**•** **The Director General of the JRC should require ISEI to audit from**
**time to time, informatics and advanced computing applications**
**throughout the** **JRC,** **against best practice standards.**

**VI.4** **Competition**

**•** **ISEI** **should be further encouraged to secure third party work.**

**•** **The rules restraining the Institute - and indeed the JRC - from**
**securing funds for participation in other CEC projects should be**
**relaxed.**

**•** **The Institute should be encouraged to adopt commercial principles**
**in bidding for third party contracts and managing such work.**

**Annex** **I** _**h**_

**Visiting Group to the Joint Research Centre's**
**Institute for Systems Engineering and Informatics (ISEI) - Ispra**

**Members**

Mr. Peter BENTON (Chairman),
Northgate House,
130 Highgate Hill,
UK-N6 6HD London.

Tel. +44-81-341.11.22
Fax; +44-81-341.11.33

(Formerly Managing Director of Telecommunications, Deputy Chairman of
British Telecom and Director General of the British Institute of Management.)

Dr. Patrice BERNARD,
Head of Department,
Department for Waste
Storageand Disposal,
Centre d'Etudes de Cadarache
Boîte postale 1,
F-13008 St. Paul lez Duras (Cedex).

Tel. +33-42-25.72.26
+ 33-42-25.21.70
Fax. +33-42-25.77.47

Dr. Gunter GRIEGER, Tel. +49-89-32.99.17.81
Director, Fax. +49-89-32.99.25.79
Max-Planck Institute
for Plasma Physics (IPP),
Boltzmannstrasse 2,
D-8046 Garching bei Munchen.

(Member of the IPP Board of Directors, Member of the IPP Scientific Board, Head
of Experimental Division.)

Dr. José Manuel TRI BOLET,
Director,
Institute for Systems Engineering

and Computers (INESC),
9 Rua Alves Redol,
P-1017Lisboa.

Tel. +351-1-31.00.333
+ 351-1-54.08.14
Fax. +351-1-57.28.97

(Founder of the Foundation for the Development of Education and Training in the
Fields of Electrical Engineering and Computer Science (FUNDETEC), Founder of
the Institute for Systems Engineering and Computers (INESC).)

Secretary

Mr. Samuel J. LLOYD,
Avenue Reine Elisabeth 8,
B-1330Rixensart.

Tel. +32-2-295.40.48 (office)
+ 32-2-654.09.44 (home)
Fax. +32-2-295.01.46

_w_

**•?>-**

Annex II

**Visiting Group** **for** **the Evaluation of the Joint Research Centre's**
**Institute** **for** **Systems Engineering and** **Informatics** **(ISEI)** **-** **Ispra**

**Scope of Visiting Groups**

The chief scope of Visiting Groups is to perform evaluations, taking into account
the objectives as set out for the 1992-1994 period (in Council documents, relevant
papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to
provide recommendations on the proper course to be followed by the Institute
visited in view of the stated objectives, and to apply in the evaluations, elements
of the methodology proposed for research returns and effectiveness.

The Visiting Group should:

perform an evaluation of all activities of the Institute against its stated
objectives, with particular emphasis on the impact the results of the work
has had on the user of this research, and the degree of collaboration
established with national research laboratories, as well as with industry;

assess the scientific quality of the work and the appropriateness of the work
and results for the Community;

judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the Community)
in comparison with similar national laboratories;

provide advice on the ongoing work on the basis of the evaluations and
assessments performed;

express recommendations for the future development of the Institute with
regard to the overall strategic plans for development of the JRC in the
medium-term and long-term.

Terms of Reference

To evaluate progress in performing work in accordance with the objectives
set out in Council Resolution 92/C118/03 and Council Decisions 92/273/EEC,
92/274/Euratom and 92/275/Euratom, documents addressed to the Board of
Governors, the JRC's Annual Workschedules, budget etc., and to review
progress in the implementation of the recommendations of the last
Evaluation Panel.

To ensure that the Institute has effective mechanisms to set its local scientific
objectives, monitor progress towards those objectives and evaluate outputs.

**?'** **2-**

**To review the various research activities to assess:**

**a)** **the** **scientific** **quality and productivity of the Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of** **funding** **and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**To assess the effectiveness of the scientific management leadership within**
**the Institute.**

**5.** **To advise on whether the Institute has been making efficient, effective and**
**economical use of resources in carrying out its programmes** **and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the higher**
**education sector, industry and government departments in the Member**
**States,** **as** **well** **as** **outside the European Community.**

**7.** **To examine the relationship of the Institute's work in general, to the mission**
**of the Joint Research Centre and to** **its** **forward strategic planning.**

**8.** **To make recommendations and report to the Board of Governors of the**
**Joint Research Centre.**

Aveiro, 22nd August 1994

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**ENVIRONMENT INSTITUTE**

_<)\_

**??**

**CONTENTS**

**I.** **Introduction**

**11.** **General Observations**

**III.** **Scientific** **and Technical Activities**

**111.1** **Air Quality**

**111.2** **Toxicology**

**III .3** **Chemicals**

**IV.** **Management of Research**

**IV.1** **General**

**IV.2** **Staff Matters**

**IV.3** **The Future**

**Recommendations**

**Annex** **I:** Members of the Visiting Group

**Annex II:** Terms of Reference of the Visiting Group

_J****_

_^AAA_

**Report of the Visiting Group** **for** **the Evaluation of the Joint Research Centre's**

**Environment Institute** **(El),** **Ispra, Italy**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), members of the Visiting Group (see Annex I) spent 26-28 April 1994
at the Environment Institute at Ispra reviewing the Institute's work and
visiting its laboratories.

The objectives for the Institute are set out in Annex I of Council Decision
92/273/(EEC) of 29th April 1992 as follows:

"The objective is to provide a contribution to the scientific knowledge,
technical know-how and data needed by the Community, in particular to
carry out its role with regard to the environment, as spelled out in Title VII
of the Treaty, and with particular emphasis on prenormative work.

Special emphasis will be placed on, in addition to the evaluation of the
effects of pollution from industrial and other sources, the assessment of
environmental degradation caused by the use of energy in the home,
transport and industry.

The JRC will contribute to the implementation of Community
environment policy in three areas:

participation in global change programmes;

technologies and engineering for the environment;

research on industrial hazards.

In the global change area, emphasis will be placed on strong cooperation
with laboratories and institutes in the Member States and at international
level. The JRC contribution will concern mainly:

modelling and air pollution transport;

air chemistry linked notably to the fate of biogenic and
anthropogenic emissions;

biosphere-atmosphere interaction;

related applications of remote sensing, which will focus on the
interactions between land and ocean surface parameters and
climate. It is furthermore aimed at the development of remote
sensing methods applied to critical components of global change.

Activities under the heading of technologies and engineering for the
environment will concern mainly:

environmental chemicals (migration and transformation in soil and
their impact on ground water, harmonization of analytical methods
for waste analysis, development of a mobile laboratory for in-field
analysis of contaminants in soil and water, characterization of algal

**/**

**P Z** **-2**

**blooms,** **biomonitoring** **of environmental** **chemicals/trace** **metal**
**exposures and human diseases, risk evaluation of environmental**
**chemicals); these activities will notably be aligned towards chemical**
**waste problems..."**

**"... In addition to research, development and support, the programme**
**would** **provide** **education and training in each of the above areas and**
**establish data collection, interpretation and distribution facilities where**
**necessary.**

**These activities could become the point of contact and common interface**
**for national organizations, in particular public authorities and industries,**
**and for support to other Commission services, including DGs** **V** **and** **XI."**

**These** **general** **objectives are concurrent with the** **JRC's** **mission and are**
**highlighted in the scope of the Visiting Groups. As set out in the terms of**
**reference (see Annex II), the Visiting Group should:**

**•** **evaluate all activities of the Institute against its stated objectives,**
**with particular emphasis on the impact the results of the work has**
**had on the users of this research, and the degree of collaboration**
**established with national research organisations, as well as with**
**industry;**

**•** **assess the scientific quality of the work and the appropriateness of**
**the work and results for the Community;**

**•** **judge all aspects of the efficiency and impact of the work, taking**
**into account the particular status of the JRC (its independence in the**
**Community) in comparison with similar national bodies;**

**•** **provide advice on the ongoing work on the basis of the** **evaluations**
**and assessments performed;**

**•** **express recommendations for the future development of the**
**Institute with regard to the overall strategic plans for development**
**of the JRC in the medium-term and long-term.**

**General Observations**

**In the three days available to review the Institute and its work, the Group**
**was impressed by the good quality of Institute management, by the**
**efforts made by staff** **anamanagement** **to prepare clear presentations and**
**documentation and to respond to questions that arose during the visit. In**
**reviewing an institute with so great a spread of activities, some selection**
**had to be made where personal visits to laboratories were concerned. All**
**activities were reviewed but only some 60% were actually visited.**

**For purposes of clarity activities were grouped into four categories:**

**Specific Programme research activities;**

**Scientific and technical activities in support of Commission policies;**

**Exploratory research work;**

**Contractual work for third parties.**

**'** **[û]**
**3-** **2>**

**For each activity, the scope, results obtained to date and likely future**
**developments were summarised in a document prepared for the visit.**
**Members of the Group appreciated this form** **of** **presentation. Members**
**were impressed by the overall quality of the science performed in the**
**Institute and of the staff who presented the various items. However, it**
**was felt that a critical process should be applied to the rather extensive**
**field covered, to identify and benefit the higher priority projects and**
**actions.**

**A characteristic of the Institute is the relatively high proportion of**
**chemists working there.** **It** **was recognised tnat in certain areas,**
**recruitment was concentrated on obtaining other disciplines and the**
**Director was strongly encouraged to continue with the estblishment of a**
**more** **multidisciplinary** **staff in the future development of the Institute. It**
**was agreed, however, that the present staff were competent and very**
**productive.**

**Another characteristic noted by the Group was the high degree of**
**geographical dispersion of the Institute throughout the Ispra site. The**
**historical** **reasons for** **this dispersion** **were** **understood** **and** **the**
**management was encouraged to pursue, with vigour, its intention to**
**regroup the Institute into a more manageable area.** **This would**
**encourage maximum synergy between the various units in the Institute**
**and thus a more systems-based approach with more extensive teamwork,**
**valuable because environmental problems are so diverse and cover the**
**vast range of human activities.**

**Following enlargement of the Union, a very strong environmental**
**research capability will be needed to respond to the growing awareness of**
**the environmental impacts of human activities and the consequent**
**evolution towards higher standards.**

**III.** **Scientific and Technical Experimental Activities**

**111.1** **AirQuality**

**The Visiting Group considered the European Reference Laboratory on Air**
**Pollution (ERLAP) to be well-managed and after ten years of operation to**
**be of proven value.**

**Emissions were still an urgent matter and the work on bio-emissions could**
**well be extented to include emission/absorption by soils. The burden of**
**pesticides, polycyclic aromatic hydrocarbons etc. in rainwater was another**
**area of relevance and work on the harmonization of parameters**
**measured and measurement methodology was of concern to everyone.**

**The Evaluation Monitoring European Pollution (EMEP) station operated**
**by the Institute was competently operated and the Group was pleased to**
**see that Ispra had an assured place in this international network.**

**A close cooperation with universities was encouraged, particularly in**
**highly specialised areas such as molecular biology and plant physiology in**
**order to establish a sound basis for the explanation of the observed**
**phenomenon on the** **"Biogenic** **Emissions in the Mediterranean Area",**
**(BEMA) for example. There is a general lack of field research institutes in**
**Europe and the work is important.**

_**A-s**_ _**[A ]**_

**The group was impressed by the degree of cooperation with Member**
**States achieved in the provision of a suite of complementary mobile**
**laboratories for on-site measurements and sampling and** **strongly**
**encouraged common actions to provide joint facilities for large-scale**
**experiments.** **For example, large-scale dispersion experiments were of**
**vital interest in understanding and predicting the probable spread of**
**contamination following industrial-scale accidents and the Institute's**
**leadership** **in this international** **cooperative** **network** **was** **warmly**
**applauded. Much effort had to be applied to all aspects of modelling in**
**an area where even relatively short-term meteorological predictions were**
**shown to be unreliable.**

**Experimental work on the behaviour of sulphur-bearing aerosols in the**
**atmosphere was under development with the commissioning of a new**
**facility and the approach of combining modelling with experimental**
**verification was sound. It appeared that in this instance, the good physics**
**should be closely linked to a more extensive chemical support to fully**
**exploit the experimental possibilities.** **The international networking**
**undertaken on these large-scale problems was appreciated and should**
**help to ensure that maximum benefit was obtained from the projet and**
**its quality assured.**

**It** **was felt that, in general, a closer cooperation between Units responsible**
**for atmospheric chemistry and physics and for biospheric-atomospheric**
**interactions would bring considerable benefits to both parties.**

**Indoor air quality was still of concern as high concentrations of many**
**products were found in domestic, public and industrial buildings. JRC**
**contributions using the INDOORTRON were significant, particularly in the**
**validation of measurement methods.** **Work in that area might be**
**completed in about 3 to 4 years which could liberate the facility for use by**
**universities and industry under contract. Round robin quality assessment**
**campaigns would be** **OT** **great value for the harmonization of measuring**
**methods in Europe, and improvement of their quality.**

**III.2** **Toxicology**

**III.2.1** **European Centre** **for** **the Validation of Alternative Methods (ECVAM)**

**Ths Centre was being set-up to meet a requirement in Directive**
**86/609/EEC that the Commission and Member States should encourage**
**research for the replacement, where possible, of animal tests by other**
**appropriate procedures. ECVAM has its own scientific advisory committee**
**which has defined the tasks as : "ECVAM will promote the scientific** **and**
**regulatory acceptance of alternative methods which are of importance to**
**the biosciences and which reduce, refine or replace the use of laboratory**
**animals".**

**A Head of Unit has been appointed, staff recruitment is under way,** **the**
**proceedings of a series of recently held workshops on current problems**
**are in press and ECVAM is at present involved in six validation studies with**
**European laboratories and with Japan and the USA.**

**The Visiting Group whilst appreciating the strong scientific guidance and**
**coordination of efforts through the Scientific Advisory Committee, felt**
**that every effort should be made to streamline administrative and**
**recruitment procedures to bring the unit up to full strength as soon as**
**possible. The Environment Institute was felt to be the most appropriate**
**EU location for ECVAM.**

**- 5 -** **/©r~**

**11.2.2** **Toxicology Laboratory**

**The Group pointed out the importance of determining background values**
**of trace metals and metal species in the general population in the**
**European Union. Such values are of importance for the evaluation of risks**
**arising from occupational exposures and those resulting from accidents to**
**chemical plant, etc...** **In** **addition, total diet studies are needed to**
**complement the above, especially in cases where vegetables are** **home-**
**grown whilst epidemiological and statistical investigations are also**
**required if results are to be applicable in practice.**

**A very important requirement lies in the standardization of all analytical**
**steps:** **sampling, sample preparation; separation; trace analysis, all of**
**which must follow agreed standard operating procedures. For parts of**
**this work, clean-room conditions and regulations are required.** **After**
**visiting the toxicological laboratories, the Group had the impression that a**
**redefinition of goals as well as a reconstitution of the Unit is necessary.**
**The present and continuing importance of chlorinated and polycyclic**
**aromatic hydrocarbons** **shoufd** **be taken into account at that time. A close**
**connection with water and soil analysis would benefit both analytical**
**services and users alike, and should be examined at an early date.**

**III.3** **Chemicals**

**111.3.1** **European Chemicals Bureau**

**This activity is typical of actions to support EU legislation and operates in**
**the area of tne control of chemicals.** **Its mission is seen to be of**
**importance for the establishment of viable regulations and directives and**
**their implementation.**

**111.3.2** **Data Banks Euclid and Ecphin**

**The data banks demonstrated to the Group appeared** **to** **be efficiently**
**conceived with attention to user-friendly interfaces and giving customer**
**satisfaction. However, the question arose as to whether, once a system**
**was developed, it should be continued by valuable institute staff. Where**
**possible, it should be passed to other private or public organizations and**
**this needed periodic review on a case-by-case basis.**

**111.3.3** **Food chemistry**

**European Office of Wine, Spirits and Spirit Drinks (BEVAPS).**

**Wine Laboratory**

**This relatively small activity provides the Commission with analytical**
**services for prevention of fraud and determination of origins of products.**
**Analytical** **methods** **are** **evaluated** **in** **cooperation** **with** **national**
**laboratories. The work appears to be competently performed. Chemical**
**and microbiological laboratories for food analysis were also visited and**
**found to be well** **equipped** **and competently staffed.** **With** **EU**
**enlargement, this could be a growth area.**

_**AP**_ _**[è ]**_

**111.3.4** **Water and Waste**

**This is a particularly important area and it is gratifying to see the JRC**
**playing its part in a range of international projects where the Institute's**
**contributions have been significant. Even so, the available resources do**
**not appear to fully reflect the** **importance** **given to these subjects in the**
**European Union.** **Water** **quality, recycling, treatment and similar**
**processes are matters likely to be raised by upcoming directives. Water**
**and waste management studies are urgently needed and waste water and**
**solid waste should be included.**

**111.3.5** **Analytical Laboratories**

**To** **conclude** **this** **section, the** **Group** **encourages** **the** **Institutes'**
**management to study the regrouping of its analytical services into a more**
**compact form: if possible, as a single unit. This would help to make the**
**most efficient use of specialized instruments and equipment and ensure**
**maximum pooling of expertise and also help to prevent possible**
**duplication.**

**IV.** **Management of Research**

**IV.1.** **General**

**The environment Institute is larger than other Institutes at** **Ispra** **with a**
**wide spread of research and service activities. With minor exceptions, it is**
**well managed by keen and enthusiastic staff but suffers from the**
**inevitable disadvantages of being widely dispersed over the Ispra site.**
**However, it is understood that relocation is planned** **for** **the coming years.**

**Evidence** **of** **networking and collaboration with other organizations in**
**Europe and elsewhere was found in each of the units visited. This is clearly**
**an important function of a Commission institute working in a field as**
**universal as the environment and the visiting group wishes to emphasize**
**the importance of trans-frontier** **cross-linking.**

**As noted before in this report, the visiting group considers that in view of**
**the importance given to environmental problems in the European Union,**
**especially after enlargement, a much more even balance of disciplines will**
**be** **needed** **in the Institute. Until that can be achieved, the further**
**development of interinstitute collaborative projects can be one method of**
**improving the spread of competences. In that respect, networking with**
**the outside world and maximising the number of post-doctoral fellows**
**and visiting scientists in the Institute assume an even greater significance.**

**The Group realizes that apart from specific programme research, the**
**Environment Institute has to provide expertise and services to a wide**
**range** **of** **Commission** **services** **responsible** **for** **developing** **and**
**implementing policies in many sectors. These tasks are essentially those of**
**the "Commission's own laboratory" but since some of this work will in**
**future be of a competitive nature, it is important that the most**
**appropriate tasks are selected relative to the policies concerned and to**
**available competences and facilities. Interviews with representatives from**
**customer** **DG's** **show a high degree of customer satisfaction and the Group**
**appreciates the efforts being made by Institute scientists to meet these**
**challenges.**

**-7**
_**,A^y-**_

**Contract research is likely to remain a very small part of the Institute's**
**activities but is both an important stimulation to research** **staff,**
**encourages contact with the outside world and is a visible demonstration**
**of competence.**

**IV.2** **Staff Matters**

**To accelerate the adjustment of the balance of competences in the**
**Institute, measures should be sought to encourage mobility. Movement**
**between customer DGs and the Institute would form closer links between**
**"contractor and client" and attention must be directed to obtaining the**
**most appropriate young recruits. The predicted growth in the Institute**
**and the age profile shown in the strategic plan for the coming years,**
**present opportunities for disciplinary diversification.**

**Concerning** **long-and** **short-term visitors to the Institute such as post-**
**doctoral researchers, visiting scientists etc, the Group welcomes the steady**
**increase in numbers over the last three years and the future intentions.**
**Maximum effectiveness of recruitment procedures is highly desirable and**
**research** **fellowships could provide a reliable** **means of selection.**
**Flexibility is very important in trying to avoid early, narrow specialisation.**

**IV.3** **The Future**

**The visiting group feels that the indicative breakdown of activity areas by**
**percentage proposed** **by the Director and given below, represents a**
**reasonable target to aim for over the coming years.**

**60%** **Support to the Commission (essentially 2nd generation)**
**20 % Large cooperative research projects (Global change, air, water, soil)**

**10%** **Data organisation and communication (like CEO, ECPHIN, aerosols**

**etc.)**
**3%** **Free (ars-pro-arte) research**
**7%** **Work for third parties**

**Predictions regarding the size of the Institute are clearly dependent on**
**available finance and overall JRC policy and therefore** **somewhat**
**uncertain. However, the target of a complement of about 300 by the year**
**2000 does not seem unreasonable in what can be considered to be an area**
**still growing in importance.**

**The above information is outlined in the Institute's strategy document**
**which is a useful tool for the development of the Institute, but is likely to**
**require some detailed revision following the present Report.**

**V.** **Recommendations**

**The primary task of the Institute should be to provide the scientific**
**support needed to underpin the preparation and implementation of**
**EU policy in the field of protection of the environment.**

**Specific research activities should preferably be performed in close**
**collaboration** **with** **others with** **the** **aim** **of** **adding** **to** **the**
**understanding of the mechanisms of large scale environmental**
**phenomena.**

_**kp**_ _**[^]**_ **-8**

**Where** **appropriate, computer** **modelling** **should** **supplement**
**experimental activities and where possible, in Union-wide networks.**

**Within the JRC and the Institute itself, a greater degree of synergy**
**between units should be established. A** **multidisciplinary** **teamwork**
**approach to large-scale problems** **is** **often the most effective one.**

**Greater diversity of staff disciplines** **is** **strongly favoured.**

**Analytical services should be reorganized to include the toxicology**
**laboratory and radioactive assay should be confined to specific**
**needs.** **Neutron activation analysis can often be substituted by more**
**modern techniques of equal or superior sensitivity.**

**Research on water and waste processing/recycling is likely to**
**increase in importance.** **Attention should be directed to these**
**aspects, particularly with regard to harmonization of methods.**

**Work on indoor air pollution should continue for some years**
**focussed on verification of standard operating procedures and**
**sampling.**

**Intermixing of staff with that of outside organizations is encouraged**
**and networking should be yet more strongly developed.**

**Data-bases should be examined for operation by outside agents,**
**only retaining those for which there are strong reasons for operation**
**by the Institute.**

**The selection of tasks and setting of priorities within the institute is**
**important. A more transparent project management system would**
**help to ensure that these are maintained, and reassure the outside**
**world of the Institute's efficiency. It should be possible to achieve**
**this without creating a top-heavy management or diverting**
**scientists from their tasks.**

Annex I _**i»§**_

**Visiting Group** **to the** **Joint Research Centre's Environment**

**Institute** **(EI)** **-** **Ispra**

**Members**

Prof. Carlos A. HOKREGO (Chairman),
Department of Environment
and Planning,
University of Aveiro,
P-3800Aveiro.

Prof. Dr. A. KETTRUP,
Head of Institute,
Institute of Ecological Chemistry,
GSF Neuherberg,
Ingoldstàdter Landstrasse 1,
D-91465 Neuherberg.

Dr. Thomas O'CONNOR,
Experimental Physics Department,
University College Galway,
IRL-Galway.

Prof. Dr. Christos ZEREFOS,
Laboratory of Atmospheric Physics,
Physics Department,
Aristotelian University of Thessaloniki,
GR-54006 Thessaloniki.

Secretary

Mr. Samuel J. LLOYD,
Avenue Reine Elisabeth 8,
B-1330Rixensart.

Tel. + 351-34-37.02.00
Fax. + 351-34-29.290

Tel. +49-89-31.87.40.47/87.40.48
Fax. + 49-89-31.87.33.71

Tel. + 353-91-24.411 ext. 2495
Fax. + 353-91-25.700

Tel. +30-31-99.27.84/99.80.41
Fax. +30-31-28.37.52/24.86.02

Tel. +32-2-295.40.48 (office)
+ 32-2-654.09.44 (home)
Fax. +32-2-295.01.46

_Jj^_

_**^AM**_

Annex

**Visiting Group for the Evaluation of the Joint Research Centre's**
**Environment Institute** **(El)** **- Ispra**

Scope of Visiting Groups

The chief scope of Visiting Groups is to perform evaluations, taking into account
the objectives as set out for the 1992-1994 period (in Council documents, relevant
papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to
provide recommendations on the proper course to be followed by the Institute
visited in view of the stated objectives, and to apply in the evaluations, elements
of the methodology proposed for research returns and effectiveness.

The Visiting Group should:

perform an evaluation of all activities of the Institute against its stated
objectives, with particular emphasis on the impact the results of the work
has had on the user of this research, and the degree of collaboration
established with national research laboratories, as well as with industry;

assess the scientific quality of the work and the appropriateness of the work
and results for the Community;

judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the Community)
in comparison with similar national laboratories;

provide advice on the ongoing work on the basis of the evaluations and
assessments performed;

express recommendations for the future development of the Institute with
regard to the overall strategic plans for development of the JRC in the
medium-term and long-term.

Terms of Reference

To evaluate progress in performing work in accordance with the objectives
set out in Council Resolution 92/C118/03 and Council Decisions 92/273/EEC,
92/274/Euratom and 92/275/Euratom, documents addressed to the Board of
Governors, the JRC's Annual Workschedules, budget etc., and to review
progress in the implementation of the recommendations of the last
Evaluation Panel.

To ensure that the Institute has effective mechanisms to set its local scientific
objectives, monitor progress towards those objectives and evaluate outputs.

**/ " •** **- 2**

3. To review the various research activities to assess:

a) the scientific quality and productivity of the Institute;

b) whether full advantage is being taken of scientific opportunities in the
light of funding and other constraints;

c) the relevance of the work with respect to Community needs, including
the relevance of work performed for external customers, when
applicable.

4. To assess the effectiveness of the scientific management leadership within
the Institute.

5. To advise on whether the Institute has been making efficient, effective and
economical use of resources in carrying out its programmes and
management functions. Resources include manpower, money, services,
facilities, data and equipment.

6. To review the extent and effectiveness of the Institute's external scientific
links, including relationships with research organisations, the higher
education sector, industry and government departments in the Member
States, as well as outside the European Community.

7. To examine the relationship of the Institute's work in general, to the mission
of the Joint Research Centre and to its forward strategic planning.

8. To make recommendations and report to the Board of Governors of the
Joint Research Centre.

##### **>n**

**Enschede, 10th December 1993**

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR**

**SAFETY** **TECHNOLOGY**

_/_ [w ]

**lu-**

**CONTENTS**

**introduction** **page** **1**

**General Observations** **page** **1**

**i.** **Scientific** **and Technical Areas**

**1.1** **Fission Reactor Safety**

**1.2** **Industrial Hazards**

**1.3** **Structural Mechanics**

**1.4** **Tritium Technology**

**1.5** **Safeguards**

**IV.** **Management of Personnel**

**IV.1** **Mission**

**IV.2** **Relations**

**IV.3** **Quality**

**pages 2-5**

**page** **2-3**

**page** **3-4**

**page** **4**

**page** **4-5**

**page** **5**

**page 5-6**

**page** **5-6**

**page** **6**

**page** **6**

**V.** **Recommendations** **page** **6**

**Annex** **I:** Members of the Visiting Group

**Annex** **II:** **Terms of Reference of the Visiting Group**

_Ii>_
/ "

_**"y**_

**Report of the Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute** **for** **Safety Technology,** **Ispra,** **Italy**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), members of the Visiting Group (see Annex I) spent the 25th, 26th and
27th October 1993 at the Institute for Safety Technology (1ST) at Ispra
reviewing the Institute's work and visiting its laboratories.

The objectives for the Institute are to be found in Annex I of Council
Decision 92/273(EEC) and 92/274/Euratom of 29 April 1992.

A. The main objectives concern:

prenormative research on the reliability of structures;
research on industrial hazards;
nuclear fission safety including reactor safety, safeguards and
fissile materials and radioactive waste management;
tritium technology and fusion safety.

These general objectives are concurrent with the JRC's mission and are
highlighted in the scope of the Visiting Groups. As set out in the terms of
reference (see Annex II), the Visiting Group should:

    - evaluate all activities of the Institute against its stated objectives, with
particular emphasis on the impact the results of the work has had on
the users of this research, and the degree of collaboration established
with national research organisations, as well as with industry;

    - assess the scientific quality of the work and the appropriateness of
the work and results for the Community;

    - judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the
Community) in comparison with similar national bodies;

    - provide advice on the ongoing work on the basis of the evaluations
and assessments performed;

    - express recommendations for the future development of the Institute
with regard to the overall strategic plans for development of the JRC
in the medium-term and long-term.

General Observations

This report is arranged by Mission area following the internal organisation
of the Institute as follows:

Fission Reactor Safety (including waste managementsSafeguards;
Industrial Hazards;
Structural Mechanics;
Tritium Technology.

_**A ^**_

**The Visiting Group examined each of these areas in turn and was impressed**
**by the leadership shown by the Institute's management team which had**
**clearly succeeded in transferring its enthusiasm to its collaborators. This**
**effect was seen to have permeated to everyone interviewed during visits to**
**laboratories and demonstrations, and was particularly evident in younger**
**staff and grantholders.** **This was manifested in an evident team spirit**
**visible at all levels.** **In this connection, the presence of junior staff at**
**meetings was much appreciated by members of the group.**

**A large measure of this success could be attributable to the highly**
**organised and efficient system of project control exercised at all levels. The**
**detailed definition of objectives, setting of milestones, analysis of strong**
**and weak points with subsequent follow-up and reporting, were important**
**elements in securing a high level of motivation and consequent cost-**
**effectiveness.**

**The re-orientation of scientists previously engaged in nuclear research and**
**the redeployment of equipment to other significant fields of R&D, was**
**considered to be clear evidence of successful management.**

**III.** **Scientific and Technical Areas**

**111.1** **Fission Reactor Safety**

**The Visiting Group considered that in view of uncertainties related to**
**national decision-making on energy policy, it would be reasonable to**
**maintain a certain nominal minimum of R&D on nuclear energy at a**
**Community level. It was pointed out that regardless of future policy, the**
**environmental implications of former and existing nuclear activities would**
**remain into the far future, both in the** **E.C.** **and in Eastern European**
**countries, with obvious implications to the Community as a whole.**

**The Visiting Group therefore, recommends that, in view of the lack of a**
**clear vision of the future, serious consideration be given to ensure that**
**work be maintained at a viable level: in some areas resources may already**
**have been reduced too far.**

_**Particular**_ _**comments**_

**•** **The combination of modelling, code development with international**
**partners, experimental verification followed by publication, was**
**considered to be a** **valid prenormative** **approach to widespread**
**coordination of high acceptability, likely to produce a maximum of**
**useful products for a minimum overall outlay in times of universal**
**financial stringency.** **The JRC is the best placed organisation to**
**motivate such networks by practical example and the activities are**
**felt to be of high importance.**

**•** **Facilities such as FARO in** **Ispra** **and PHEBUS (in Cadarache) and the**
**future STORM were of real importance in the area. Members were**
**pleased to see that doubts expressed in the report issued by the 1991**
**Evaluation Panel, chaired by Sir Hermann Bondi, about a long-term**
**future for FARO had been dispelled by outside support for the**
**projects being performed there.**

**'?**

**The Group regretted** **that** **the PETRA facility could not be brought into**
**operation, feeling that such a facility could be of great help in** **aiding**
**common waste treatment problems. However, in view of restricted**
**funds and the present lack of national interest** **in** **research on**
**reprocessing, the decision to proceed slowly was understood.** **In**
**nuclear** **fission** **safety,** **the** **general** **lack** **of** **on-site** **projects**
**necessitating the handling of radioactive substances was noted.**

**No evidence was seen of work leading to inherently safe nuclear**
**plants,** **a subject of world interest in which JRC expertise could**
**contribute usefully.**

**Industrial Hazards**

_**Comments**_

**Since experimental and theoretical work on dispersions of heavy gas**
**clouds is carried** **on** **in many places worldwide, the members felt that**
**there was no special place for the** **1ST** **in that area. However, if the**
**Institute endeavoured to include chemical reactions in dispersion**
**models,** **so** **as to** **incorporate** **ignition** **of** **fires,** **explodable**
**deflagrations and detonations in a sound way, this would represent a**
**difficult and major step forward in science, and enable a valid**
**judgement to be made of the environmental and safety aspects of**
**certain measures taken by local, national or international authorities.**

**In the chemical industry most accidents occur under dynamic**
**conditions, such as those experienced during start-up and shut-down**
**of large continuously operated plants** **andf** **in batch or semi-batch**
**processing of chemical reactions.** **On the basis of the** **IST's** **long**
**experience in dynamic studies and multiphase fluid flow,** **the**
**selection of safety in chemical plants and the environmental impact of**
**accidents as a new research project was a wise and** **useful** **choice.**

**The combination of the study and growing know-how in the**
**experimental and theoretical evaluation of venting facilities, pressure**
**relief, handling of severe accidents and dispersion of toxic chemicals**
**in the environment is unique.** **The developing collaboration with**
**industrial partners concerning pressure relief and venting is highly**
**relevant and should be pursued vigorously.**

**The** **experimental** **installation** **(FIRES) for** **studying** **dangerous**
**reactions is fully up to an international standard and the** **1ST** **is very**
**well equipped for the collection of generic, fundamental know-how**
**applicable to all chemical plants.**

**Knowledge of chemical reactions and chemical reactors themselves is**
**still at an initial stage in the institute. We strongly recommend hiring**
**a few experienced chemical engineers or industrial chemists to build**
**up that** **knowledge** **as quickly as possible and to provide adequate**
**sparring partners for industry.**

**Industrial enterprises will only study reactions appropriate to their**
**own processes and consequently there is a great lack of fundamental**
**knowledge** **on chemical** **plant dynamics.** **Universities find** **it**
**impossible to maintain groups with requisite experience on a**
**permanent basis and the institute should fill this gap.**

_**a-**_

**•** **The** **1ST** **should strengthen its contacts with other institutes in Europe**
**also working in the area of batch reactors, to set up fruitful**
**cooperations and thus establish its own niche in research in this field.**
**This should be fundamental technological research of a generic**
**nature which can help all industrial companies and simultaneously**
**provide the European Commission with a sound technological**
**background for developing directives.**

**.3** **Structural Mechanics**

**The activity centres around experiment design and model development**
**using numerical and visualisation techniques, followed by experimental**
**verification using the recently commissioned European Laboratory for**
**Structural Assessment (ELSA) and the Large Dynamic Testing Facility (LDTF).**

_**Comments**_

**•** **The Group noted with satisfaction the excellent progress made with**
**the development of activities in the Unit and the enthusiasm and**
**developing competence of the staff. First tests in ELSA have been very**
**satisfactory.**

**•** **A potential study of considerable interest exists in the investigation of**
**the connections between pre-cast concrete elements together with**
**the diaphragmatic behaviour of slabs and** **reinforced** **masonry under**
**seismic conditions.**

**•** **While** **recognising** **the** **importance** **of** **a** **common** **computer**
**environment, the Visiting Group considers that the Unit should resist**
**the temptation to disperse its activities in too many fields of**
**application and concentrate its talent on fewer selected topics where**
**an in-depth approach would bring greater rewards.**

**.4** **Tritium Technology**

**In the use of tritium for civil purposes it must be emphasised that due to its**
**nature, the behaviour of the element is very different from that**
**experienced with fission products. This has important implications, at a**
**European level, on protection and safety. It is therefore necessary to gain**
**experience in the handling of large quantities of tritium before fixing the**
**engineering design of future thermonuclear fusion reactors and further**
**large-scale experiments incorporating tritium.**

_**Comments**_

**•** **The Group visited the European Tritium Handling Laboratory ETHEL,**
**at present being commissioned, and was impressed by the progress**
**made so far towards the practical implementation of a working**
**laboratory.**

**•** **Initial pilot experiments are progressing well and the experimental**
**programme is sound.**

**•** **Members were pleased to find a close collaboration with the**
**installation at Karlsruhe and noted the complementarity between the**
**work carried out** **inthe** **two laboratories..**

**f*(**

**Excellent progress is being made overall on these very important**
**activities**

**III.5** **Safeguards**

**This JRC activity is not only an important link in the worldwide nuclear**
**safeguards network and the implementation of non-proliferation policy,**
**but provides the essential technical support required by the Commission in**
**fulfilling its statutory obligations under Chapter** **VII of** **the EURATOM**
**Treaty.**

_**Comments**_

**•** **The Visiting Group was pleased by the efficiency and care apparent in**
**the performance of the work and its management.** **However,**
**Members were concerned at the lack of a clear management structure**
**to operate the programme which is spread over a number of JRC**
**Institutes.** **They believed that although no adverse effects had**
**appeared up to the present, attention should be paid to this point.**

**•** **The long standing training programme for Safeguards inspectors**
**carried out in the PERLA facility was particularly impressive and an**
**important contribution to the effective safeguarding of nuclear**
**materials in Europe.**

**•** **The steady development** **of field** **instrumentation** **to** **improve**
**accuracy, portability, means of standardisation and user-friendliness**
**was considered to be of** **importance** **and to be performed to a high**
**standard.**

**In view of recent events demonstrating the need for an accurate and**
**convenient means of assaying the quality of the contents of waste**
**containers, the Visiting Group welcomed the response made by the**
**JRC in developing rapid and highly sensitive equipment for waste**
**container monitoring. The equipment appeared to be precise and**
**largely automatic in use.**

**•** **The Visiting Group was pleased to see that support was also provided**
**to inspectors from the IAEA, Vienna and that the JRC took the lead in**
**the European Safeguards Research and Development Association**
**(ESARDA)** **and** **had worldwide** **connections** **in the** **matter** **of**
**safeguarding nuclear materials.**

**IV.** **Management of Personnel**

**IV.1** **Mission**

**•** **Objectives were found to be generally clear and appropriate to the**
**JRC and the project structure installed in the Institute defined**
**activities in all mission areas and fixed mission targets and time scales.**

**•** **The evolution of current Community research policy in the energy**
**field is a cause for concern in avoiding** **thatthe** **effort is not decreased**
**below a critical mass due to reductions in resources.**

_**H**_ **-** **[6]** **-**

**IV.2** **Relations**

**Information transfer, inward and outward, appeared to be good in all**
**sectors covered by the Institute, largely due to the Institute's**
**involvement with other organisations in Member States and outside**
**by means of working parties, collaborations and initiating** **workshops,**
**as** **well** **as** **a number of joint projects.**

**It** **was noted that the volume of contract work undertaken for third**
**parties** **had** **doubled in the last year and the Visiting Group would**
**encourage further efforts in this respect.** **Such contracts were a**
**valuable means of demonstrating the Institute's vitality and**
**competence to industry and would help in securing** **IST's** **place in the**
**scientific world.**

**IV.3** **Quality**

**The Institute has been able to maintain a generally high standard in**
**most areas of work, in spite of a steady conversion process of staff**
**moving from mainly nuclear to other tasks. This was due to the**
**quality and flexibility of a motivated staff.**

**V.** **Recommendations**

**The Institute should continueto follow and develop the lines being**
**pursued at present paying attention to avoid too rapid a**
**diversification** **beyond the areas** **of** **existing competences.**

**Attention should be paid to the maintenance of a suitable level of**
**resources in the nuclear safety area.** **Possible needs arising from**
**Eastern European developments should not be neglected.**

**The feasibility of the Institute's involvement in work on inherently**
**safe nuclear reactors should be examined.**

**The embryonic Industrial Hazards activities are worth pursuing**
**vigorously and should be supplemented with an injection of**
**experienced industrial chemical engineers or chemists.**

**The structural mechanics team should be careful not to spoil an**
**excellent start by too wide a dispersion of activities. This applies**
**particularly to analytical modelling.**

**The already close cooperation in tritium technology with other**
**laboratories active in the world should be maintained to ensure that**
**best use** **is** **made of this important Community facility.**

**The Group feels that exploratory research could be a management**
**tool for** **finding** **new and relevant activities to further enhance the**
**future standing of the institute.** **Such programmes should not**
**require prior approval but** **should** **be** **judgea** **a** _**posteriori**_ **as to**
**whether management has spent this money diligently.**

**The excellent system of project control employed by the Institute**
**could well be extended to embrace pan-JRC projects such as nuclear**
**safeguards which spans several institutes.**

**I*)**
**Annex**

**Visiting Group to the Joint Research Centre's**
**Institute for Safety Technology** **(1ST)** **- Ispra**

**Members**

**Prof.** **Dr.** **Franz** **M.** **BAUMGAERTNER,** **Tel.**
**Director,** **Fax.**
**Institute for Radiochemistry,**
**Technical University of Munich,**
**3 Walther Meissner,**
**D-8046 Garching.**

**(Areas of expertise: Tritium uptake in plants; aerosol**
**chemistry; analysis at low-level transuranium elements.)**

**+ 49-89-32.09.22.02**
**+ 49-89-32.61.115**

**Prof.** **Dr. Theodossios P.** **TASSIOS,**
**Agias Lavras 4,**
**GR-15236 Palea** **Penteli (Athens).**

**Tel. +30-1-362.48.40 (office)**
**+ 30-1-804.97.51 (home)**
**Fax. +30-1-362.54.82**

**(Areas of expertise: The behaviour of materials and structural engineering**
**under seismic conditions and environmental influences.)**

**Prof.** **Dr.** **Ir.** **Roel** **WESTERTERP (Chairman),** **Tel.**
**Head of Department,** **Fax.**
**Department of Industrial Processes,**
**P.O.** **Box** **217,**
**University of Twente,**
**NL-7500AEEnschede.**

**(Areas of expertise: 25 years in production, R&D, sales**
**and general management in the chemical industry.)**

**Secretary**

**+** **31-53-89.28.77**
**+ 31-53-31.40.24**

**Mr. Samuel J. LLOYD,**
**Avenue Reine Elisabeth 8,**
**B-1330Rixensart.**

**Tel. +32-2-295.40.48 (office)**
**+ 32-2-654.09.44 (home)**
**Fax. +32-2-295.01.46**

/ _[iH ]_

_**SÀ1<**_

Annex II

**Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute** **for Safety** **Technology** **(1ST)** **- Ispra**

**Scope of Visiting Groups**

The chief scope of Visiting Groups is to perform evaluations, taking into account
the objectives as set out for the 1992-1994 period (in Council documents, relevant
papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to
provide recommendations on the proper course to be followed by the Institute
visited in view of the stated objectives, and to apply in the evaluations, elements
of the methodology proposed for research returns and effectiveness.

The Visiting Group should:

perform an evaluation of all activities of the Institute against its stated
objectives, with particular emphasis on the impact the results of the work
has had on the user of this research, and the degree of collaboration
established with national research laboratories, as well as with industry;

assess the scientific quality of the work and the appropriateness of the work
and results for the Community;

judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the Community)
in comparison with similar national laboratories;

provide advice on the ongoing work on the basis of the evaluations and
assessments performed;

express recommendations for the future development of the Institute with
regard to the overall strategic plans for development of the JRC in the
medium-term and long-term.

Terms of **Reference**

To evaluate progress in performing work in accordance with the objectives
set out in Council Resolution 92/C118/03 and Council Decisions 92/273/EEC,
92/274/Euratom and 92/275/Euratom, documents addressed to the Board of
Governors, the JRC's Annual Workschedules, budget etc., and to review
progress in the implementation of the recommendations of the last
Evaluation Panel.

To ensure that the Institute has effective mechanisms to set its local scientific
objectives, monitor progress towards those objectives and evaluate outputs.

**-**
_**p6**_ **_** **2**

**3.** **To review the various research activities to assess:**

**a)** **the scientific quality and productivity of the Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of funding and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management leadership within**
**the Institute.**

**5.** **To advise on whether the Institute has been making efficient, effective and**
**economical use of resources in carrying out its programmes and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the higher**
**education sector, industry and government departments in the Member**
**States,** **as** **well as outside the European Community.**

**7.** **To examine the relationship of the Institute's work in** **general,** **to** **the mission**
**of the Joint Research Centre and to its forward strategic planning.**

**8.** **To make recommendations and report to the Board of Governors of the**
**Joint Research Centre.**

**Bonn,** **2nd September, 1994**

**REPORT** **OF** **THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR REMOTE**

**SENSING APPLICATIONS**

_A_ _[%\ ]_

_**^**_

**CONTENTS**

**I.** **Introduction**

**II.** **General Observations**

**III.** **Scientific and Technical Activities**

11.1 Agricultural Information Systems (AIS)

11.2 Environmental Mapping and Modelling (EMAP)

11.3 Monitoring Tropical Vegetation (MTV)

11.4 Marine Environment (ME)

1.5 Advanced Techniques (AT)

II.6 Centre for Earth Observation (CEO)

**IV.** **Management Aspects**

IV.1 Staff

IV.2 Visitors

IV.3 Finance

IV.4 Management Information Tools

**Annex** **I:** Members of the Visiting Group

**Annex II:** Terms of Referenceof the Visiting Group

**Annex** **III:** Institute Objectives

**Annex IV:** Programme of the Work for the Visiting Group

/ _[v> ]_

**^ M**

**Report of the Visiting Group for the Evaluation of the Joint Research Centre's**
**institute for Remote Sensing Applications** **(IRSA),** **Ispra,** **ttaly**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), members of the Visiting Group (see Annex I) spent 23-24 August
1994 at the Institute for Remote Sensing Applications (IRSA) at Ispra
reviewing the Institute's* work. The composition of and Terms of
Reference for the Visiting Group are given in Annexes I and II. The
objectives for the Institute are listed in Annex III. The Visiting Group
considers these objectives outdated and recommends a revision along the
lines given in Chapter II (in italics). **(Recommendations are highlighted in**
**bold type in the Report).**

IRSA conducts research and pre-operational system development in the
fields of:

Agricultural Information Systems (AIS);

Environmental Mapping and Modelling (EMAP);

Monitoring Tropical Vegetation (MTV);

Marine Environment (ME);

Advanced Techniques (AT);

as well as in the area of data pre-processing and dissemination through a
project called "Centre for Earth Observation (CEO)" which was recently
transferred to the Institute.

II. **General** Observations

The Visiting Group had the opportunity to listen to several presentations
by IRSA senior staff on the major topics of work and was able to visit a
number of laboratories and facilities.

The presentations were generally of good standard evidencing the
professionalism and good success of the Institute in the few years since its
creation in 1988. There seems to be a good level of motivation amongst
the approximately 90 permanent and 50 non-permanent staff.

11.1 Future Strategy

The Institute's **draft strategic plan** of 18th March 1994 seems to reflect a
general uncertainty about the proper way forward. The plan must be
revised and restructured towards realistic and concrete long term goals.
**These goals must in priority represent the thematic orientation of the**
**Institute's activities and be described in measurable goals such that**
**implementation plans** **can** **be drawn up aqainst these goals.** The
indiscernible mixture of strategic, tactical and operative aspects of the
present draft plan should be eliminated.

                        - 2

The Visiting Group took the opportunity to analyse the present **objectives**
**of the Institute** and found them outdated. It recommends reformulations
along the following lines which should on the one hand help to better
focus the Institutes activities and (on the other) pay tribute to the
important and further growing role of the Institute within the JRC:

_**(i)**_ _**to**_ _**evaluate**_ _**and**_ _**demonstrate**_ _**applications**_ _**of**_ _**remote**_ _**sensing**_
_**techniques in support of the sectorial policies**_ _**of**_ _**the European Union**_
_**in areas such as agriculture,**_ _**environment,**_ _**aid to**_ _**development,**_
_**regional**_ _**aid...**_

_**(ii)**_ _**to contribute**_ _**towards the support of the European scientific and**_
_**application**_ _**communities in the utilisation**_ _**of earth**_ _**observation**_
_**satellite data.**_

_**Note: This objective is complementary to those of the European and**_
_**national**_ _**space agencies, international**_ _**scientific programmes**_ _**and**_
_**those of other institutions;**_

_**(Hi)**_ _**to conduct research and support the development**_ _**of methods and**_
_**advanced**_ _**techniques for the interpretation**_ _**and application**_ _**of**_
_**satellite derived data.**_

The strategic plan should further clarify the goals and policy in the
following fields:

roles and relationships with national, European and international
organisations - where a clear understanding should be the baseline
of the Institute's activities. The subsidiarity of the Institute's
contribution is an important consideration in establishing an
unambiguous position in this respect. At the same time, attention
must be focused on fixing priorities and the development of a policy
on work that should, respecting subsidiarity, be performed in-house
and what tasks can be better placed _extra-muros._ This should help
towards clarifying IRSA's policy on third party activities and its
competitive role _vis-à-vis_ national research organisations and
industry;

data processing and dissemination - where the Institute should make
additional contributions towards the **promotion and furtherance of**
earth observation in the different communities;

education **and** training - where the activities of the Institute should
**be strengthened** by e.g. further enlarging the number of visiting
scientists or cooperation with academia, young graduate trainee
programmes and the like.

There is an urgent need to further clarify the **Institute's strategic and**
**operational roles and share** of **activity in the EC's Fourth Framework**
**Programme.** The results of these ongoing efforts will have an important
effect on the Institute's orientation and activities for the next four to six
years and must be reflected in the updated strategic plan.

3

Quality

The quality and the dedication of the **acting Director and senior staff** are
recognisea by the Visiting Group. It is recommended that a decision on
the appointment of a Director to lead the Institute is taken in the shortest
possible time. This should be coupled with a requirement to improve the
**application of generally accepted academic standards** to the activities and
the external presentation of the Institute.

The Group felt that although the Institute enjoys considerable success,
efforts should be made to improve its impact by greater attention to
detail and verification of information in botn oral and written
presentations.

It should be also evaluated whether the setting up of **additional advisory**
**groups,** representing the user communities, would be the appropriate
step to enhance guidance and acceptance of the Institute's work.

Methods for the Dissemination of the Institute's Activities' Results

In general, the Institute develops and verifies methods for the
interpretation of aerospace data for application in a number of fields by
organisations responsible for policy implementation as mentioned above.
The product in that case is passea to the organisation concerned and a
follow-up service is provided. This service is particularly appreciated by
the relevant Commission Directorates General and services.

There is also a steady contribution of articles in journals, presentations to
international meetings, however, the Group was concerned to find a
number of papers listed as "submitted" which do not appear in
subsequent publication lists. This practice is felt to be **misleading and**
**likely to engender** **misunderstandings** **in the academic world.**

Dissemination by direct transfer to students, post graduate and post
doctorate research fellows is a growing feature at IRSA. Participation in
large projects with other organisations by networking is an important and
developing means of two-way information transfer. At the same time, a
**more** active **approach to third party contract research,** particularly in the
Advanced Techniques area, should also help with information
dissemination and the development of techniques in Europe.

**III.** Scientific and Technical Activities

1 Agricultural Information System (AIS)

The main objectives of the Agricultural Information Systems Unit are to
provide scientific and technical support to the services of the community
in charge of the execution of the Common Agricultural Policy (CAP). The
central core of the activities of the AIS are the Monitoring Agriculture
with Remote Sensing (MARS) project and the Register and Control of
Agricultural Surfaces project, both taking the form of **a** technical and
scientific support provided to the Directorate General VI (Agriculture).

**33**

                        - 4

This unit has shown its deep understanding of the issues in agriculture
information problems and a remarkable scientific maturity in its approach
to the use of space remote sensing in this field. In undertaking and
conducting the corresponding activities it followed the appropriate
strategy:

support to research in order to establish the methods;

validation of the scientific methodology and clear statement of the
limits and accuracy of the method imposed by the physics of the
measurements;

transfer of the scientific method and its adaptation as an operational
method;

validation of the operational method;

industrial transfer in order to develop the operational tool and
implement it;

transfer of the operational tool under the responsibility of the user
(DGVI).

The AIS Unit is well managed and is characterised by a good scientific and
technical guidance. It is effective in establishing innovative operational
tools in agricultural information for the benefit of the Community Policies
and requiring familiarity with modern techniques in data acquisition
process and physical analysis of data. The Visiting Group noted with
satisfaction the quality of cooperation reached with scientific and
industrial teams of the Member States during the development of the
project.

In view of the importance of the agriculture information issue both from
the political and economical sides, the Visiting Group encourages the unit:

(a) **to extend the method** it developed for statistics in agriculture in the
European Union **to other regions** (Central and Eastern Europe, North
Africa ...);

(b) to develop the methodologies for the control of agricultural
surfaces;

(c) to expand the programme towards the **assessment of agricultural**
productivity and crop estimations for Europe and in the long term
for other parts of the world.

Environmental Mapping and Modelling (EMAP)

The research in EMAP concerns the mapping of various components of the
European landscape and the development of methods for using remotely
sensed data together with other geocoded information for the
management and protection of the European environment. During 1993,
the mapping and monitoring activities of the Unit were conducted along
t w o main thematic lines:

(i) forest grassland and natural vegetation;

(ii) Mediterranean land degradation.

-5

Parallel activities in image understanding were pursued, aimed at
continuing the development of methodologies for automatic
classification, mapping and map generation from remote sensing data.

The continued cooperation with Central and Eastern European countries
(through the PHARE regional programme) should go on a par with the
**development of an interest for remote sensing data from a larger range**
**of satellites (e.g. Russian).**

The Unit has the project of extending its work to "coastal ecosystem
monitoring". It should be ensured **how this will be coordinated with** **the**
**Marine Environment Unit and integrated in the Land Ocean Interaction**
**system in the Coastal Zone** **(LOICZ)** **and the European Land Ocean**
**Interaction StudiEs (ELOISE) Programme.**

The Visiting Group noted with surprise the **lack of clear goals and**
**objectives from the Commission's environmental programmes and**
**recommends the clarification of such goals and the interaction of the**
**EMAP Unit with the Environmental Agency and DG XI.**

III.3 Monitoring Tropical Vegetation (MTV)

The objectives of the Monitoring Tropical Vegetation (MTV) Unit, closely
connected to the objectives of the International Biosphere Geosphere
Programme (IGBP), concern the study of vegetation at the surface of the
continents using satellite based earth observations (development of
methods for monitoring vegetation dynamics, derivation of physical
characteristics of land surfaces and interface this information with climate
models). The emphasis is put on the study of the tropical belt as
containing the most actively changing world ecosystems, and the
phenomena and processes are considered at global scales.

The subject studies concern the land cover and land use characterisation
(Mercator Project), the monitoring of forest and biomass burning
(TREES/FIRE), surface-atmosphere interactions and biosphere modelling
(TEAM project).

The activities of this Unit are carried out under a good scientific guidance,
a specific and successful effort to associate European scientists and to link
the subject study to the objectives of the international scientific
programmes on global change.

The members of the Unit often participate in international^ research
groups and maintain a regular dialogue with the main space agencies
which provide space systems or instruments dedicated to vegetation
monitoring. It is well worth noting that the MTV Unit, in the frame of its
programme devoted to scientific research and methodological
developments, had the opportunity to contribute directly to the DG I and
DG VIII actions for the benefit of developing countries.

The Visiting Group appreciates the quality of the produrts developed by
the Unit and the efforts aimed at maintaining a high degree of external
collaboration. It considers that one of the important achievements is the
development of a comprehensive forest information system in support of
the research community modelling the tropical deforestation dynamics.

The Visiting Group encourages the Unit to work towards the setting up of
**an operational information system and entity to monitor in the long term**
**the global vegetation dynamics.**

_**I**_ _**M**_

_Jli4_

111.4 Marine Environment (ME)

The scope of the Marine Environment Unit is to develop, demonstrate and
validate methodologies for the use of data from space and airborne
observations in both operational applications and scientific investigations
related to the marine environment.

The Unit is basically involved in the study of t w o main remotely sensed
parameters:

(i) the ocean colour as an expression of the substances in the near
surface water layers and the biological activity in the euphotic zone;

(ii) the sea surface temperature as an indication of the heat energy
budget at the air-sea interface as an element of the general
hydrodynamic processes.

Several publications in international journals with referees testify to the
Unit's activity.

The Unit has developed a unique finalised product and it should be
recommended that **methods and means be found to make it available, on**
**a routine basis, to the** **océanographie** **community.**

Under a formal inter institute agreement the Unit is responsible for the
development of hydrodynamic models with the aim of assimilating
remote sensing data in process studies and of contributing to global
models. Although this is a natural development of the remote sensing
studies, **one can not help wondering whether the Institute** **-** **even with the**
**reinforcement from other Institutes of the JRC - has a staff of sufficient**
**size** or **expertise to pursue successfully such an ambitious project.**

It is recommended that the Unit adopt as one of their long term thematic
goals the development of **an operational tool for regional planners for**
**the application of coastal ocean management.**

111.5 Advanced Techniques (AT)

The Advanced Techniques Unit is the biggest inside the IRSA, staff 40
persons. The Unit is responsible for development of techniques and
methods needed in scientific and application oriented use OT remote
sensing. This includes e.g. the investigation of the principles of
measurements in the different spectral bands, investigation of the
feasibility of different techniques, development of data analysis methods
and tools, and development of information extraction methods.

The main elements of the activities are research on optical remote sensing
systems, development of signal processing techniques, use of the
European Microwave Signature Laboratory (EMSL), and use of the
European Airborne Remote Sensing Capabilities (EARSEC).

The AT Unit is well managed and the Visiting Group got the impression
that the staff is very well qualified and motivated in their work. The
Group was impressed with the high quality of the equipment and
installations. Especially the European Microwave Signature Laboratory is
an installation which permits remarkable contributions to the science and
to the development of practical applications of remote sensing (signature
research).

_^ 1 5 ^_

The AT Unit has an Advisory Group to follow and make recommendations
on the work of the Unit. The Visiting Group found this good practice.

The Visiting Group draws the Institute's attention to the need to further
**promote the excellent capabilities of the** **Unit** **and to encourage the**
**acquisition of third party** **financing** **and work** **for** **its facilities.**

111.6 Centre for Earth Observation (CEO)

The Visiting Group was exposed to a concise presentation of the objectives
of and plans for the Centre for Earth Observation (CEO). It was based on
documents previously presented to the JRC's Board of Governors and
discussed in the Steering Committee of the Pathfinder phase of the CEO,
nominated by this Board. This presentation started with an overview of
the situation in the field of space-based earth observations by the end of
the century (provision of very important volumes of data, addressing the
needs of a variety of end users in the scientific and applications fields),
recalled the roles of the major actors in the field OT data processing,
distribution and use (EC, ESA, Member States and other European relevant
organisations) and presented the CEO main objectives:

provide user oriented services;

be constructed on existing facilities and projects;

be designed to be flexible;

be designed to promote earth observations in Europe.

The CEO schedule and workplan were also presented and in particular its
Pathfinder Phase, covering the period 1993-1995. The Pathfinder Phase is
composed of five main activities leading to the detailed definition and
cost estimates of the CEO, in preparation of the decision on its design and
implementation.

The Visiting Group expressed its satisfaction to see the positive evolution
of this extremely important project for the user of earth observation space
data in the European Union. The CEO will play a major role in the
promotion and dissemination of data and information, in stimulating
applications and creating new products.

The Visiting Group stressed the fact that the **CEO should be a really**
**decentralised system, designed for the end users, taking intp account**
existing European and national facilities. The Pathfinder Phase should in
particular aim at clarifying the objectives and roles of the CEO with
respect to research and development of new applications in support,
among others, of the policy of the EU. The Visiting Group is confident in
the success of the endeavour undertaken under the responsibility of the
IRSA, with the cooperation and advice of European scientists, engineers
and managers.

It is recommended that a more suitable **name be found for this activity.**

**434**

                                         - 8 

IV. Management Aspects

IV.1 Staff

The Institute is favoured with a relatively young staff with an average age
of 44. However, it is clear that in a rapidly evolving field such as remote
sensing, there will be a need for a career planning particularly at the
younger end of the age spectrum. At the same time, temporary
attachment of staff to customer DGs can help to give a certain number of
people a clear understanding of user need as well as bringing notions of
remote sensing capabilities to the organisation concerned.

Staff recruitment appears to leave much to be desired with a top heavy
bureaucratic system bogged down with complicated procedures and long
delays. **Simplification** **should be made wherever possible.**

IV.2 Visitors

The presence of a fairly high number of post doctoral and post-graduate
research fellows is an important element in bringing scientific vitality to
Institute permanent staff, as well as the future advantage of a corps of
scientists throughout Europe with an intimate knowledge of thé IRSA and
advanced techniques of modern earth observation. Training young
scientists is in itself an important activity in such an Institute. This **activity**
should be expanded.

IV.3 Finance

The Visiting Group noted with satisfaction the generally good situation
with respect to the future years' financing of the Institute's staff but
noted with some concern the difficulties that the Institute is faced with in
respect of maintaining an adequate budget for financing external
activities. Under the system of competitive research via Shared Cost
Actions and competitive bidding for contracts to provide support to DGs a
number of difficulties will have to be overcome. **Some** income **from** these
sources will be needed if the Institute is **not to shrink.**

Furthermore, there is a perceived conflict between the subsidiarity
requirement and the need to compete for further work with national
institutes and research organisations. **Careful attention** must be given to
this matter if actual conflict is to be **avoided.**

IV.4 Management Information Tools

As a consequence of the above and of the need for a greater perceived
transparency, a more formalised **project** **management** information system
**should be** established **for the whole Institute,** compatible with other JRC
Institutes.

**Annex** **I** _**if**_

**Visiting Group to the Joint Research Centre's**
**Institute for Remote Sensing Applications (IRSA) - Ispra**

**Members**

**Professor Jacques** **NIHOUL,**
**Geo Hydro Dynamics and**
**Environmental Research (GHER)**
**Bâtiment** **B5,**
**Université de Liège,**
**SartTilman,**
**B-4000 Liège.**

**Dr. Isaac** **REVAH,**
**Director of International Relations,**
**Centre National** **des**
**Etudes Spatiales (CNES),**

**2 Place Maurice Quentin,**
**F-75039** **Paris** **Cedex** **1.**

**Prof.** **Heinz STOEWER (Chairman),**
**Managing Director,**
**Space Programmes,**
**Deutsche Agentur fur Raumfahrt-**
**angelegenheiten (DARA) GmbH,**
**Konigswinterer Strasse 522-524,**
**D-53227Bonn.**

**Professor. Seppo URPO,**
**Director,**
**Metsâhovi Radio Research Station,**
**Helsinki University** **of** **Technology,**
**Otakaari 5A,**
**SF-02150Espoo.**

**Tel.** **+32-41-56.33.50**
**Fax. +32-41-56.23.55**

**Tel.** **+33-1-44.76.75.85/44.76.76.28**
**Fax. +33-1-44.76.78.49**

**Tel.** **+49-228-44.74.20/44.74.23**
**Fax. +49-228-44.77.06**

**Tel.** **+358-0-451.2235/264.831**
**Fax. +358-0-460.224/264.541**

**Prof.** **Dr. Caesar VOUTE*,** **Tel.** **+31-53-87.42.16/87.44.44**
**P.O.** **Box 1079,** **Fax. +31-53-87.44.00**
**NL-7500BBEnschede.**

*unable to attend meeting but provided written comments on documentation

**Secretary**

**Mr. SamuelJ.** **LLOY1),**
**Avenue Reine Elisabeth 8,**
**B-1330Rixensart.**

**Tel. + 32-2-295.40.48 (office)**
**+ 32-2-654.09.44 (home)**
**Fax. +32-2-295.01.46**

_/_ _[4o ]_

_**,yiv(**_

**Annex II**

**Visiting Group** **for** **the Evaluation of the Joint Research Centre's**
**Institute** **for** **Remote Sensing Applications (IRSA) - Ispra**

**Scope of Visiting Groups**

**The chief scope of Visiting Groups is to perform evaluations, taking into account**
**the objectives as set out for the 1992-1994 period (in Council documents, relevant**
**papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to**
**provide recommendations on the proper course to be followed by the Institute**
**visited in view of the stated objectives, and to apply in the evaluations, elements**
**of the methodology proposed Tor research returns and effectiveness.**

**The Visiting Group should:**

**perform an evaluation of all activities of the Institute against its stated**
**objectives, with particular emphasis on the impact the results of the work**
**has had on the user of this research, and the degree of collaboration**
**established with national research laboratories, as well as with industry;**

**assess the** **scientific** **quality of the work and the appropriateness of the work**
**and results for the Community;**

**judge all aspects of the efficiency and impact of the work, taking into**
**account the particular status of the JRC (its independence in the Community)**
**in comparison with similar national laboratories;**

**provide advice on the ongoing work on the basis of the evaluations and**
**assessments performed;**

**express recommendations for the future development of the Institute with**
**regard to the overall strategic plans for development of the JRC in the**
**medium-term and long-term.**

**Terms of Reference**

**To evaluate progress in performing work in accordance with the objectives**
**set out in Council Resolution** **92/C118/03** **and Council Decisions 92/273/EEC,**
**92/274/Euratom and 92/275/Euratom, documents addressed to the Board of**
**Governors, the** **JRC's** **Annual Workschedules, budget** **etc.,** **and to review**
**progress in the implementation of the recommendations of the last**
**Evaluation Panel.**

**To ensure that the Institute has effective mechanisms to set its local scientific**
**objectives, monitor progress towards those objectives and evaluate outputs.**

**N***

**3.** **To review the various research activities to assess:**

**a)** **the scientific quality and productivity of the Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of funding and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management** **leadership** **within**
**the Institute.**

**5.** **To advise on whether** **the** **Institute has been making efficient, effective and**
**economical** **use of** **resources in carrying** **out** **its programmes** **and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the** **higher**
**education sector, industry and government departments in the Member**
**States,** **as well as outside the European Community.**

**7.** **To examine the relationship of the Institute's work in general, to the mission**
**of the Joint Research Centre and to** **its forwardstrategic** **planning.**

**8.** **To make recommendations** **and** **report to the Board of Governors of the**
**Joint Research Centre.**

**^3**

Annex

**INSTITUTE OBJECTIVES**

The Institute has four main objectives:

1. To evaluate and demonstrate possible applications of remote
sensing techniques in support of the sectorial policies of the
European Community in areas such as:

agriculture;
environment;

aid to development;
regional aid.

2. To prepare the scientific community for the utilisation of earth

observation satellites such as:

ERS-1;

SPOT;

Landsat;

NOAAAVHRR;

Polar Orbiting Platforms.

This objective is complementary to that of the European Space
Agency.

To undertake research in order to develop methods for the
interpretation of satellite derived data.

To develop advanced techniques of earth observation for
application in the mapping and monitoring of continental and
marine ecosystems.

yjw4 - . \ A <

London, 19th November 1993

**REPORT OF THE VISITING GROUP**

**FOR THE EVALUATION OF THE**

**JOINT RESEARCH CENTRE'S**

**INSTITUTE FOR PROSPECTIVE**

**TECHNOLOGICAL STUDIES**

yW*

**WÇ**

**CONTENTS**

**I.** **Introduction**

**11.** **Work in Accordance with Objectives**

**Efficiency**

**IV.** **Overall Conclusions**

**V.** **Recommendations**

**Annex** **I:** Members of the Visiting Group

**Annex II:** Terms of Reference of the Visiting Group

_**A**_

**Report of the Visiting Group for the Evaluation of the Joint Research Centre's**
**Institute for Prospective Technological** **Studies,** **Ispra,** **Italy**

**Introduction**

At the invitation of the Board of Governors of the Joint Research Centre
(JRC), the Visiting Group (see Annex I for Members) spent 8th and 9th
September 1993 at the Institute for Prospective Technological Studies at
Ispra. The Group met again in Brussels on 22nd October 1993 to examine its
findings and meet with further clients from the Commission's services.

The objectives for the Institute are to be found in the Technical Annex to the
JRC Programme Proposal 1992-1994 (COM(91)281 final). The principal tasks

are:

1. The acquisition, treatment and analysis of information concerning the
state and trends of science and technology.

2. The execution of prospective studies in targeted scientific and
technical (S & T) areas.

To perform these tasks, the Institute has two main functions. The first of
these is the observatory function in which the Institute endeavours to:

   - collect information on trends in technological innovation;

   - analyse, process and present the information in order to help
determine research and technological development strategies;

   - develop and use specialised databases of the information collected by
the Institute;

   - describe the state of science and technology in selected areas of
interest to the Community.

The second function is to execute prospective technological studies. This
involves preparing S & T assessments, forecasts and scenarios, as an aid to
the formulation of research and development strategies, including analyses
of economic, social and environmental impacts.

In pursuit of both functions, the Institute will make the best possible use of
existing national bodies active in the same area, organising European
networks of S & T observatories and prospective studies units.

These general objectives are concurrent with the Joint Research Centre's
mission and are highlighted in the scope for the Visiting Groups. The
Visiting Group should:

   - perform an evaluation of all activities of the Institute against its stated
objectives, with particular emphasis on the impact the results of the
work has had on the users of this research, and the degree of
collaboration established with national research organisations, as well
as with industry;

   - assess the scientific quality of the work and the appropriateness of the
work and results for the Community;

_**Àïl-**_ **-2**

               - judge all aspects of the efficiency and impact of the work, taking into
account the particular status of the JRC (its independence in the
Community) in comparison with similar national bodies;

               - provide advice on the ongoing work on the basis of the evaluations
and assessments performed;

               - express recommendations for the future development of the Institute
with regard to the overall strategic plans for development of the JRC in
the medium-term and long-term;

This report will be organised following the order set out in the Terms of
Reference (see Annex II) with the Group's recommendations for each point
summarised under "Conclusions".

**II.** **Work in Accordance with Objectives**

               - Owing to the structure of JRC funding under the Framework
Programme, specific research programme funds could not be made
available to the Institute's activities. As a consequence, the Institute
has had to rely on customer demand for its resources and little money
has been allocated for the development of the "Observatory" as a unit.
However, some of the functions of an observatory are found
associated with the supply of information for other projects and
activities such as Global Change and the C0 2 issue, and Best Available
Technology (BAT) fall into this category;

               - for the same reasons, work in the technological forecasting area is
related to customer requirements and largely comprises state-of-theart extrapolations;

               - following demand, a different role has developed/that of Scientific
and Technological Advisory Unit: the principal customers being the
Secretariat General's Forward Studies Unit (CDP), DG III (Industry),
DG XI (Environment, Nuclear Safety and Civil Protection), DG XVII
(Energy) and DG Xll's aeronautical programme;

               - 90% of the Institute's work is in support of the Commission.

_Conclusions_

               - The Institute has not met the objectives originally foreseen for it; but,

               - it has evolved an important role of Scientific and Technological
Advisory Unit which should be developed;

               - the observatory function has lagged behind for reasons linked with
financing, but could now develop;

               - the role of the Institute needs to be further formalised so that its work
can be fully integrated with policy formation.

**-3** **i*3**

**III.** **Efficiency**

**The Visiting Group was able to examine the output in reports and other**
**publications but was not** **allowed** **to remove these for detailed perusal for**
**reasons of confidentiality.** **Output was also assessed from documentary**
**evidence provided by clients** **and** **by interviewing representatives of**
**Commission services which use the Institute.**

**•** **At the time of the visit, 40 documents had been produced for the**
**Commission and 2 for external third parties;**

**•** **scientific and technical staff** **totalled** **14, including** **11** **JRC officials (10**
**university and one technical grade), 2** **seconded** **experts and one**
**visiting scientist, together with 8 grantholders, of which 3 are post**
**doctorate, 3 are preparing PhDs and 2 are pursuing post graduate**
**specialisation;**

**•** **the Group was impressed by the commitment and enthusiasm of the**
**staff.**

_**Conclusions**_

**•** **Considering the number of staff, funds available and the very positive**
**reactions from the clientele where the science and technology advisory**
**function is concerned, the output is considered to be most satisfactory;**

**•** **unavailability of the output for expert evaluation, particularly in cases**
**where this represented compilations of existing information in the**
**public domain, was considered to be a barrier to making the Institute**
**better known and obtaining maximum public benefit from the**
**Institute's results;**

**•** **quality was not evaluated in detail although this area of work had**
**been assessed by** _**ad hoc**_ **expert groups chosen for each project.**

**IV.** **Overall Conclusions**

**•** **The main activities of the Institute were fulfilling an important**
**function as a Science and Technology Advisory Unit; ***

**•** **in general, the Institute was not yet filling its designated role of**
**technological forecasting and the** **'Observatory"** **function was only**
**developing slowly;**

**•** **the quality of output could not be assessed from documents shown to**
**the Visiting Group as they were not made available for more study or**
**peer review off-site;**

**•** **customer reactions, both verbal and written, were appreciative of the**
**Institute's contribution to their own work.** **The convenience of**
**working with an in-house team acquainted with Commission culture**
**and practice was considered to be a positive factor. For certain clients**
**such as the Forward Studies Unit, the confidentiality assurance of**
**working with a Commission group was also important, as was the**
**impartiality of the advice given;**

_**<H**_

**present core activities concerned scientific and technological aspects of**
**energy, environment and transport, and their interactions;**

**accountability within the JRC parent organisation was unclear to the**
**Visiting Group;**

**the Visiting Group noted that the Institute would be transferred to**
**Seville in the near future and expressed concern about retaining**
**efficient working relationships and the necessary day-to-day contacts**
**with the Commission in Brussels. The S & T Advisory Unit should be**
**located in Brussels;**

**work undertaken up to the present was felt to be appropriate for**
**meeting the Commission's needs. Work for outside clients was judged**
**to fall within the Institute's competence, more effort should be applied**
**to the development of these activities and building collaborative**
**arrangements with third parties.** **There had been little work for**
**industrial clients so far;**

**there was a clear distinction between the work carried out by the**
**Institute and that undertaken by other Commission bodies involved in**
**the field such as the Forward Studies Unit,** **SAST,** **SPEAR and forward**
**looking technology units in other Directorates General, however, in**
**the case of FAST some redefinition of the interface should be made;**

**it was noted that all important studies are submitted to review panels;**

**in most cases, the value added by the work lies in integrating, collating**
**and interpreting complex S & T information rather than adding**
**knowledge.**

**Recommendations**

**•** **The JRC, in view of its flexibility and in-house competences, appears to**
**be the most suitable organisation to house such an institute;**

**•** **accountability within the JRC management as the parent organisation**
**should be clarified to ensure maximum transparency of operations**
**since the Institute is financed mainly from public funds and much of**
**the Institute's output can be considered to fall within the public**
**domain.** **This would help** **to** **create a healthy attitude inside the**
**Institute and a better understanding of the Institute's purpose and**
**achievements in the outside world;**

**•** **the tasks of the Institute should consist of four basic activities which**
**provide:**

**Research and Development (R & D) in Focused and Specific Fields**
**and Operation of an Information Infrastructure (Observatory):**

**The R & D activity should be focused on a small number of fields**
**in which the Commission has policy interests. These fields are**
**likely to comprise** **activities** **bridging the interests of more than**
**one policy sector, such as transport, energy, environment and**
**informatics,** **where** **inter-relationships** **need** **to** **be** **better**
**understood.** **The information infrastructure should establish**
**well-focused data based on available technologies and "best**
**practices" in well-selected technology fields.**

**. 5 -** **t>**

**Training**

**The Institute should pay particular attention to training activities**
**which are considered to be very important, both to train analysts**
**and researchers, and to set up** **"natural"** **networks in** **the** **future;**

**Commercial and Regional Services**

**The Institute should also undertake regional tasks as they arise,**
**and work under contract for third parties should be developed as**
**a substantial and** **meaningful** **part of the activity;**

**Scientific** **and Technological Advice**

**Science and Technology (S & T) are important factors to be**
**considered in many areas of policy.** **A focus for S & T advice**
**should be provided by a small, but competent, Unit of the**
**Institute. To be effective, this Unit should be located in Brussels.**
**The role of the Unit should be to respond and interact with the**
**policy formation process and to be aware of current policy issues**
**and give advice as appropriate.** **A real and ongoing need has**
**been identified for this support.**

**To accomplish this task, the Brussels Unit of the Institute should**
**draw on the best expertise available from a wide spectrum** **of**
**interests including the other parts of the Institute.** **The Unit**
**should be closely and flexibly linked with "think thanks" in the**
**Commission.**

**The Institute's S & T Advisory Unit should not** **only** **respond on**
**request, but also advise on** **S** **& T matters where this can usefully**
**inform Commission policy makers. A level of about 6 staff in**
**Brussels** **is** **felt to be** **appropriate.**

**it is considered essential to maintain a clear distinction between the**
**Institute's work and the responsibilities of other Commission think**
**tanks and forward studies groups. Nevertheless, strong links should be**
**forged with these units;**

**in view of its important advisory tasks, the whole Institute must employ**
**staff of the highest quality. Where practical, recruitments rather than**
**transfers of other JRC staff should prevail when setting up the Institute**
**in Seville. Some 50% of the scientific manpower could consist of non-**
**JRC** **staff;**

**to achieve credibility in the scientific world and ensure accountability,**
**the research findings should be published wherever possible;**

**annual** **progress** **reports** **should** **include** **a** **section** **on** **the**
**implementation of these recommendations and a further evaluation**
**by a Visiting Group should be envisaged during 1997 to monitor**
**progress. In the meantime, the detailed implementation plans for the**
**whole Institute should be approved by the Board of Governors of the**
**Joint Research Centre.**

**r**

_kOo_

**Annex** **I** _**[JCf-]**_

**Visiting Group to the Joint Research Centre's**
**Institute for Prospective Technological Studies (IPTS) - Ispra**

**Members**

**Dr.PeterE.deJÂNÔSI,** **Tel. +43-2236-715.220**
**Director,** **+43-2236-715.402**
**International Institute for** **Fax. +43-2236-731.47**
**Applied Systems Analysis** **(IIASA),**
**A-2361** **Laxenburg.**

**Sir John FAIRCLOUGH** **(Chairman),** **Tel. +44-71-280.59.30**
**Chairman,** **Fax. +44-71-283.02.42**
**Rothschilds Ventures Limited,**
**New Court,**
**St.** **Swithin's** **Lane,**
**UK-EC4P4DU** **London.**

**Dr.FriederMKYKR-KRAHMER,** **Tel. +49-721-680.91.02**
**Director,** **+49-721-680.91.03**
**Fraunhofer Institut fur Systemtechnik** **Fax. +49-721-68.91.52**
**und Innovationsforschung,**
**Breslauerstrasse 48,**
**D-7500 Karlsruhe 1.**

**Secretary**

**Mr. Samuel J. LLOYD,** **Tel. +32-2-295.40.48 (office)**
**Avenue Reine Elisabeth 8,** **+32-2-654.09.44 (home)**
**B-1330Rixensart.** **Fax. +32-2-295.01.46**

Vs _si_

**n**

**Annex II**

**Visiting Group** **for** **the Evaluation of the Joint Research Centre's**
**Institute** **for** **Prospective Technological Studies (IPTS) - Ispra**

**Scope of Visiting Groups**

**The chief scope of Visiting Groups is to perform evaluations, taking into account**
**the objectives as set out for the 1992-1994 period (in Council documents, relevant**
**papers for the Board of Governors, JRC Annual Workschedules, budget, etc.), to**
**provide recommendations on the proper course to be followed by the Institute**
**visited in view of the stated objectives, and to apply in the evaluations, elements**
**of the methodology proposed** **Tor** **research returns and effectiveness.**

**The Visiting Group should:**

**perform an evaluation of all activities of the Institute against its stated**
**objectives, with particular emphasis on the impact the results of the work**
**has had on the user of this research, and the degree of collaboration**
**established with national research laboratories, as well as with industry;**

**assess the scientific quality of the work and the appropriateness of the work**
**and results for the** **Community;**

**judge all aspects of the efficiency and impact of the work, taking into**
**account the particular** **status of** **the JRC (its independence in the Community)**
**in** **comparison** **with similar national laboratories;**

**provide advice on the ongoing work on the basis of the evaluations and**
**assessments performed;**

**express recommendations for the future development of the Institute with**
**regard to the overall strategic plans for development of the JRC in the**
**medium-term and long-term.**

**Terms of Reference**

**To evaluate progress in performing work in accordance with the objectives**
**set out in Council Resolution** **92/C118/03** **and Council Decisions 92/273/EEC,**
**92/274/Euratom and 92/275/Euratom, documents addressed to the Board** **of**
**Governors, the** **JRC's** **Annual Workschedules, budget etc., and to review**
**progress in the implementation of the recommendations of the last**
**Evaluation Panel.**

**2.** **To ensure that the Institute has effective mechanisms to set its local scientific**
**objectives, monitor progress towards those objectives and evaluate outputs.**

**2-**

**3.** **To review the various research activities to assess:**

**a)** **the** **scientific** **quality and productivity of the Institute;**

**b)** **whether full advantage is being taken of scientific opportunities in the**
**light of funding and other constraints;**

**c)** **the relevance of the work with respect to Community needs, including**
**the relevance of work performed for external customers, when**
**applicable.**

**4.** **To assess the effectiveness of the scientific management leadership within**
**the Institute.**

**5.** **To advise on whether the Institute has been making efficient, effective and**
**economical** **use of** **resources in carrying** **out** **its programmes** **and**
**management functions.** **Resources include manpower, money, services,**
**facilities, data and equipment.**

**6.** **To review the extent and effectiveness of the Institute's external scientific**
**links,** **including relationships with research organisations, the** **higher**
**education sector, industry and government departments in the Member**
**States,** **as well as outside the European Community.**

**7.** **To examine the relationship of the Institute's work in general, to the mission**
**of the Joint Research Centre and to its forward strategic planning.**

**8.** **To make recommendations and report to the Board of Governors of the**
**Joint Research Centre.**

_J6À_

**ISSN 0254-1475**

**COM(95)** **60 final**

# **DOCUMENTS**

**EN** **01 15**

**Catalogue** **number** **:** **CB-CO-95-072-EN-C**

**ISBN** **92-77-86186-X**

**Office for Official Publications of** **the European Communities**

**L-2985** **Luxembourg**