Special report No 9/2001 on the Training and Mobility of Researchers Programme together with the Commission's replies
Date: 2001-07-19
Institutions: European Court of Auditors
Subjects: EU programme, job mobility, report, research staff, vocational training

10.12.2001

EN

Official Journal of the European Communities

C 349/1

I

(Information)

COURT OF AUDITORS

SPECIAL REPORT No 9/2001

on the Training and Mobility of Researchers Programme together with the Commission’s replies

(pursuant to Article 248(4)(2), EC)

(2001/C 349/01)

CONTENTS

Paragraph

Page

LIST OF ABBREVIATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Research activities in the field of the training and mobility of researchers . . . . . . . . . . . . . .

Objectives and budget of the TMR programme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The Court’s audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SELECTION OF PROPOSALS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Audit approach and general results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Research networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Access to large scale facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Contract extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

CONTRACTS AND MANAGEMENT CONTROL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Research networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Access to large scale facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Training through research (Marie Curie Fellowships) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Follow-up of previous observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Decentralisation of selection and management of Marie Curie Fellowships . . . . . . . . . . .

Taxation of Marie Curie Fellows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Marie Curie Fellowship Association . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Accompanying measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

On-the-spot controls by the Commission . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

I-VII

1-8

1

2-7

8

9-15

9-11

12

13-15

14-15

16-31

16-18

19-22

23-28

23

24

25

26-28

29-30

31

SURVEY RESULTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

32-36

3

4

5

5

6

7

7

7

7

8

8

8

8

9

9

9

9

9

9

10

10

10

C 349/2

EN

Official Journal of the European Communities

10.12.2001

Paragraph

Page

CONCLUSIONS AND RECOMMENDATIONS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Selection of proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Contracts and management control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Selection of proposals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

37-50

37-40

37

38-40

41-50

41

Contracts and management control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

42-50

The Commission’s replies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

12

12

12

12

12

12

12

14

10.12.2001

EN

Official Journal of the European Communities

C 349/3

LIST OF ABBREVIATIONS

TMR

Training and Mobility of Researchers

DG RESEARCH

Directorate-General for Research

4FP

MCFA

IHP

Fourth Framework Programme

Marie Curie Fellowship Association

Improving Human Potential

CORDIS

Community Research & Development Information Service

C 349/4

EN

Official Journal of the European Communities

10.12.2001

SUMMARY

I. This report analyses the quality of the Commission’s management systems used for pursuing the Training
and Mobility of Researchers (TMR) programme as part of the Fourth Framework Programme (1994-1998). It
concentrates on the procedures for selecting proposals and the management systems for monitoring contracts
(see paragraphs 1 to 8).

In the evaluation of proposals the procedures followed sometimes lacked transparency: there was an
II.
absence of detailed records of the deliberations leading to the decisions made. In the allocation of proposals to
evaluators potential conflicts of interest may arise (see paragraphs 9 to 15). The Court recommends that the
procedures in place should be improved and rigorously followed (see paragraph 41).

III.
In the case of the research networks activity, organisations which were not applying the eligibility criteria
for researchers were found. Individuals who had already received fellowship awards from the Commission to
execute a project were recruited (see paragraphs 16 and 17). The Commission should carry out cross-checks
to ensure these instances do not recur (see paragraph 42).

IV. Weaknesses were found in the procedures for establishing unit costs in the access to large scale facilities
activity. Only scientific officers from the Commission, and not financial officers, are evidenced as involved with
the contractors in establishing daily user fees (see paragraph 21). Advice from financial officers should be
sought and documented before the user fees are agreed (see paragraph 44). In addition, the external mid-term
review was carried out at an inopportune time (see paragraph 22). This review should be carried out in line
with the actual contract duration (see paragraph 45).

V. The success rates for proposals for Marie Curie Fellowships submitted varied significantly according to the
submission procedure adopted by the prospective fellow (see paragraph 24). The management of the Marie
Curie Fellowships should be centralised in the department involved in the TMR programme where significant
improvements in management have occurred (see paragraph 46).

VI. The Commission does not systematically check that all the criteria it establishes for the euroconferences,
practical training courses and summer schools are being respected. In addition, no information is obtained on
sponsorship received from other entities for these events (see paragraphs 29 and 30). Procedures should be
altered to ensure these checks are carried out (see paragraph 49).

VII. From the results of the survey carried out it can be noted that the beneficiaries/contractors are on aver-
age satisfied with the service they receive from the Commission (see paragraphs 32 to 36).

10.12.2001

EN

Official Journal of the European Communities

C 349/5

INTRODUCTION

Research activities in the field of the training and mobility of
researchers

1. The aim of the Community’s research policy, as laid down in
Article 163 of the Treaty, is to strengthen the scientific and tech-
nological bases of Community industry and encourage it to
become more competitive at international level. Community strat-
egy in the field of research and technological development is laid

down in framework programmes and implemented by the Com-
mission through specific programmes. The Fourth Framework
Programme (1) (1994-1998) is still being implemented while the
Fifth Framework Programme (2) (1998-2002) has commenced
following the European Parliament and Council Decision of
22 December 1998. In the Fourth Framework Programme (4FP)
the ‘stimulation of the training and mobility of researchers’ was
selected as one of the four activities for which 792 million euro
were allocated out of a total budget for the Framework Programme
of 11 879 million euro (see Table 1). The Council adopted a deci-
sion (3) on ‘a specific programme of research and technological
development, including demonstration, in the field of training
and mobility of researchers’.

Breakdown by measure of the Fourth Framework Programme’s budget

Table 1

Fourth Framework Programme

(Mio EUR)

Activity 1

Activity 2

Activity 3

Activity 4

Research, technological development and demonstration programmes

A. Information and communication technologies
B. Industrial technologies
C. Environment
D. Life sciences and technologies
E. Non-nuclear energy
F. Transport
G. Targeted socioeconomic research

3 668,0
2 140,0
1 157,0
1 709,0
1 076,0
263,0
147,0

Cooperation with third countries and international organisations

575,0

Dissemination and optimisation of results

352,0

Stimulation of the training and mobility of researchers

— Research networks
— Access to large scale facilities
— Training through research (Marie Curie Fellowships)
— Accompanying measures

Source: European Commission.

Total

10 160,0

575,0

352,0

Grand total

11 879,0

357,0
119,0
277,0
39,0

792,0

(1) Decision No 1110/94/EC of the European Parliament and of the
Council of 26 April 1994 concerning the fourth framework pro-
gramme of the European Community activities in the field of research
and technological development and demonstration (1994 to 1998)
(OJ L 126, 18.5.1994). Decision No 2535/97/EC of 1 December 1997
adapting Decision No 1110/94/EC for the second time (OJ L 347,
18.12.1997).

(2) Decision No 182/1999/EC of the European Parliament and of the
Council of 22 December 1998 concerning the fifth framework pro-
gramme of the European Community for research, technological
development and demonstration activities (1998 to 2002) (OJ L 26,
1.2.1999).

(3) Council Decision 94/916/EC of 15 December 1994 adopting a spe-
cific programme of research and technological development, includ-
ing demonstration, in the field of training and mobility of research-
ers (1994 to 1998) (OJ L 361, 31.12.1994).

C 349/6

EN

Official Journal of the European Communities

10.12.2001

Objectives and budget of the TMR programme

2. The TMR programme is divided into three interdependent
areas (research networks, access to large scale facilities, training
through research), to which is added an activity concerning accom-
panying measures aimed at improving communication between
researchers themselves and with industry, at encouraging young
researchers and at publishing the results and achievements of the
programme.

3. Research networks aim to allow researchers from five or more
research teams in at least three countries to pool their efforts in a
common research project and to constitute, in this manner, groups
capable of performing research of higher quality. The access to
large scale facilities activity is devoted to large scale installations
whose uniqueness or rarity in the Community, high investment
or upkeep costs, and importance for research justify substantial
effort at Community level. Use of these facilities is funded to the
extent that researchers from other states can obtain access to the
installations that was previously unavailable. The aim of the train-
ing through research activity (which was subsequently renamed
Marie Curie Fellowships) is to allow European researchers to
undergo training or specialisation outside their country of origin.

4. Accompanying measures are taken to contribute to the objec-
tives of the programme in relation to the stimulation of researcher
mobility and training while being at the same time an appropri-
ate vehicle for the dissemination of information on the conditions
and procedures for participating in the various activities of the
programme and for the dissemination of its results. These are
referred to as euroconferences, practical training courses and
summer schools.

5. The TMR programme is managed by a unit of four depart-
ments comprising 59 members of staff as at 31 December 1999
in DG Research.

6. The budget for the four main areas in this programme was
amended following the accession of Austria, Finland and Sweden.
Table 1 shows the budget split between these areas. The total
amounts committed and paid each year can be found in Table 2.
The main point derived from these figures relates to a certain
under execution of the payment appropriations in the first years
of the 4FP.

7. Table 3 shows the amount of the TMR budget used in admin-
istrative and personnel expenditure from 1995 to 1999. The
Council Decision sets a ceiling on this expenditure of 5,6 % of the
total budget. It can be seen that the Commission is within these
limits.

Implementation of the budget — Training and mobility of researchers

Table 2

Areas

1995

1996

1997

1998

1999

2000

(Mio EUR)

Commitment appropriations
Payment appropriations
Actual commitments
Actual payments

Source: European Commission.

114,55
24,50
106,91
13,66

268,54
137,66
259,47
116,18

173,10
135,47
154,11
102,97

220,87
176,08
220,06
172,62

—
129,36
—
121,15

—
78,20
—
72,75

10.12.2001

EN

Official Journal of the European Communities

C 349/7

Training and mobility of researchers — Administrative expenditure

Table 3

Area

1995

1996

1997

1998

1999

(Mio EUR)

Commitments (1)
— Personnel
— Administration

Payments (1)

— Personnel
— Administration

3,41
3,12

6,53

3,36
2,30

5,66

Total

Total

3,66
4,05

7,71

3,71
3,93

7,64

4,11
4,49

8,60

4,10
4,19

8,29

4,29
4,49

8,78

4,28
3,65

7,93

Commitments as a percentage of total commitment appropriations

Payments as a percentage of total payment appropriations

(1) Including contributions from EFTA/EEA.
Source: European Commission.

—
—

—

0,28
0,15

0,43

3,99 %

3,78 %

The Court’s audit

8. This audit was focused on analysing the management systems
used by the administrative services both at the Commission and
at contractor level in pursuing the programme and on examining
the quality of the way they were managed. The main audit objec-
tive was to assess the effectiveness and efficiency of the TMR pro-
gramme. The procedures in place for selecting contracts were
audited and a follow-up was carried out on the weaknesses noted
in the earlier report by the Court (1) in this area. Ten on-the-spot
audits were carried out in six Community countries, during the
course of which 19 programme beneficiaries involved in 51 con-
tracts were audited. Finally, by means of on-the-spot audits and a
postal survey, information on both the contractors/beneficiaries
involved in the programme and the level of satisfaction with the
service provided by the Commission was obtained.

SELECTION OF PROPOSALS

Audit approach and general results

10. When a proposal is submitted, as a first step it is reviewed
by between two and five evaluators (called Primary Reviewers)
selected by the Commission. A preliminary single overall score is
then awarded which is normally the average of the scores awarded
by the Primary Reviewers. As a second step the Primary Review-
ers then meet (called a panel meeting) to discuss the results of the
evaluations. In the panel meeting the preliminary single overall
score can change in line with specific written procedures. The
final list produced by the panel meeting is submitted to the Com-
mission. In the cases examined by the Court, the summary min-
utes produced, however, do not include the deliberations leading
to the production of this final list. The final decision on which
proposals to accept is taken by the Commission.

11. There were some variations in the way in which, in practice,
proposals were evaluated between the four activities. Weaknesses
were found in all areas, with findings being that no documents
were produced explaining the evaluators’ decisions and that evalu-
ators had assessed proposals to which they were linked; cases
were found where an evaluator assessed a proposal in which his/
her employer/association was involved.

9. Proposals received under specific calls for proposals were
audited to determine if the written procedures were correctly fol-
lowed and if the actual selection processes were transparent. Pro-
posals are submitted by organisations and individuals throughout
the Member States and associated States. Success rates vary
between areas and calls.

(1) Annual Report (Chapter 9) concerning the financial year 1994,
together with the institutions’ replies (OJ C 303, 14.11.1995).

Research networks

In the area of research networks, the ranking stated in the
12.
summary minutes of the panel meeting differed from the single
overall scores stated in this document. The proposals were assessed
in accordance with the procedure in force, but this is not fully
reflected in the minutes in so far as the panel’s proceedings are
only partially documented. The minutes record the adjustments

C 349/8

EN

Official Journal of the European Communities

10.12.2001

made by the Primary Reviewers in order to ensure, where neces-
sary, that their individual scores are consistent with various pro-
posals before the plenary discussion. The minutes do not, how-
ever, record the subsequent adjustment made to the score of
certain proposals in the light of the panel’s plenary discussion.
This leads to an apparent inconsistency between the final classi-
fication, resulting from the panel’s discussions, and the individual
scores of some proposals. There is, in any event, a certain lack of
rigour in the minutes, but this in no way impairs the assessment
result (list of proposals which the panel suggests that the Com-
mission finance, in order of priority). A total of 13 proposals were
funded by the Commission as a result of the recommendations of
this panel.

In the analysis of the proposals in 1995 and 1997 for the
15.
above contracts, it was noted, in 27 of the 38 proposals, that one
of the evaluators of each proposal was the same in both periods.
In one of these cases, the same two evaluators assessed the pro-
posals in 1995 and 1997 (in 1997 a third evaluator was involved).
From the analysis of the relative annual reports on these contracts
for the years 1996 to 2000, it was noted that in 11 instances one
of the evaluators was linked to the contractor. In seven instances
the evaluator was a referee in the contractor’s user selection panel
and in the other four cases the evaluator was a user of the facility,
and therefore a beneficiary of the Commission funding. In one
particular case both 1997 evaluators were using the facility they
assessed in 1997 and 1998, and 1997 and 1999 respectively.
Indeed, these evaluators were benefiting from the funded access
to this facility in 1997 in the four weeks around the time they car-
ried out the evaluations. One of them had to interrupt his stay at
the facility to attend the panel meeting in Brussels. All of these
cases could lead to conflicts of interest which the Commission
procedures should prevent.

Access to large scale facilities

13. For the area of access to large scale facilities, in 13 cases out
of 61, the explicitly detailed instructions given for determining the
ranking of the proposals were not respected. This resulted in 10
proposals being given higher preliminary rankings than the assess-
ments indicated. Six of these 10 proposals were funded. No docu-
ments were found justifying these higher rankings.

Research networks

CONTRACTS AND MANAGEMENT CONTROL

Contract extensions

14.
In this area, the third call for proposals in 1997 resulted in
new proposals being received from existing contractors, request-
ing additional funds for granting access. Thirty-eight of these were
successful. Instead of issuing new contracts for these proposals,
contract amendments were signed extending the duration of the
existing initial contracts and the financial contribution. This
approach was taken to ensure the available budget was used
up. In future, the Commission should ensure this situation does
not recur and it should obtain written advice from its legal service
on the correct procedures to apply. In addition, two different
forms of overlap were found with these contracts. The first con-
cerns the duration of the contracts, where between one and nine
months of the time period of the first contract was also consid-
ered part of the extension in duration of the amended contract.
This overlap occurred to avoid ‘new contracts’ commencing after
the end of 4FP. The second instance of overlaps concerns the
additional financing, whereby in some instances cost statements
covering periods in the initial contracts were financed from the
additional funding for future periods. This situation was not
caused by the contractors but as a result of a misinterpretation of
the contract on the part of the Commission services involved. The
total amount paid to the contractors over the duration of the con-
tract was correct but, because of the above misinterpretation,
some amounts were prematurely paid.

16.
In the research networks activity (also known as research
training networks), various conditions apply to the personnel
costs of visiting researchers. Two of these conditions are that:

(a) his/her appointment must be temporary and for a fixed-term
starting after the commencement date of the contract, and

(b) he/she must not be a national of the State in which the con-
tractor appointing him/her is situated and he/she must not
have carried out his/her normal activities in that State for
more than 18 of the 24 months prior to his/her appointment.

17. During the on-the-spot audits, instances were found where
researchers who had previously received fellowships were declared
as visiting researchers on the research network even though they
had spent up to 36 months prior to their appointment in the
same organisation. These researchers were thus not eligible to be
classed as visiting researchers.

18. A common complaint during the on-the-spot audits and
from the survey was that the contractors found it difficult to
recruit researchers for the project. Many had misunderstood the
instructions and thought they were limited to advertising on Cor-
dis and had very few applications. In addition many contractors,
not just those in the research networks activity, were critical of
the Cordis website itself. It was considered too voluminous and
difficult to use and many organisations avoided using it if pos-
sible.

10.12.2001

EN

Official Journal of the European Communities

C 349/9

Access to large scale facilities

Training through research (Marie Curie Fellowships)

In the model contract for the access to large scale facilities
19.
activity, the allowable costs are split into two categories for the
cost statements: travel and subsistence costs and user fees.

20. The travel and subsistence costs may be charged to the con-
tract in accordance with the ‘normal internal rules and proce-
dures of the contractor’. Therefore, if the systems in place at the
contractor are inadequate, and since there is an absence of mini-
mum requirements requested by the Commission, it is extremely
difficult to check the validity of these costs. Instances were found
where supporting documentation was not available to justify
expenditure. In addition, the absence of verification from the users
of the facilities that they did not receive reimbursement for their
travel and subsistence costs from their home organisation led to
instances of double financing. A university professor has been
brought before a disciplinary committee as a direct result of the
above situation.

21.
In the majority of these contracts, user fees per day were
agreed between the Commission and the contractor. These were
established in ecu per day. The exchange rate exposure of the
Commission was therefore limited and cost statements were easier
to review as the rates had been agreed in advance. However, no
evidence was found that the advice of financial officers had been
sought by the scientific officers in determining these user fees. As
the user fees absorbed a high portion of the funds (in some
instances up to 95 % of a contract with a maximum Commission
contribution of 2 million ECU), it was imperative that they were
established at a fair, accurate level. The rules for calculating the
user fees per day specify that all costs should be calculated in
accordance with accounting principles relating to historic costs.
The audits identified that, while the majority of contractors had
declared the actual costs in determining the user fees, a small
number had used incorrect calculations. For example, for a con-
tract at the Joint Research Centre, the calculation of the user fees
per day was actually based on estimates, resulting in user fees in
the contract more than 30 % higher than could be justified using
historic costs. DG Research is currently investigating this.

Follow-up of previous observations

23. Since the Court’s previous report on this area, efforts have
been made to improve the management of this activity. One of
the main criticisms concerned the absence of follow-up by the
Commission when scientific reports are not received from fel-
lows. Substantial improvements have been noted in this area with
payments being linked to receipt of reports and automatic follow-
ups being sent by the Commission to contractors when reports
are not received.

Decentralisation of selection and management of Marie
Curie Fellowships

24. Marie Curie Fellowships are available not just to applicants
in the TMR programme but also under specific calls for specific
thematic programmes existing in the Fourth Framework Pro-
gramme. In the analysis of the evaluations of Marie Curie Fellow-
ships the average success rate of applications in the TMR pro-
gramme was found to vary considerably from that in the thematic
programmes. In one instance, the divergence was so wide that a
candidate would have been three times more likely to succeed
with an application in the context of a specific programme than
in the TMR programme.

Taxation of Marie Curie Fellows

It was noted that the application of income tax to the sti-
25.
pend of Marie Curie Fellows varied from country to country and
even between tax districts in the same country. The non-
harmonisation between tax districts in a country makes the pro-
cess of determining monthly stipends for each fellow visiting that
country difficult for the Commission and causes significant varia-
tions in the net amount received by the fellows in these countries.

Marie Curie Fellowship Association

22. A requirement of the contract is that mid-term reviews be
carried out on all contracts over 1 million euro in this area. This
review was carried out on all contracts regardless of value in
1998. For the contracts that commenced in 1998, the review
came too early as many were only operational for two to three
months and were merely evaluating requests for access. For other
contracts ending in 1998, the review came too late as it was not
possible to improve areas criticised before the contract ended. The
only contracts suited to the timing of this review were the 38 con-
tract extensions mentioned in paragraphs 14 and 15.

26. The Marie Curie Fellowship Association (MCFA) was estab-
lished in November 1997 as an international, non-profit-making
association under Belgian Law. The aim of the MCFA, as defined
in its statutes, is to:

(a) help advance science in Europe;

(b) create a clear and visible identity for Marie Curie Fellowships;

C 349/10

EN

Official Journal of the European Communities

10.12.2001

(c) promote and increase the reputation of Marie Curie Fellow-

ships;

(d) improve contact and the flow of information between the

members of the Association;

It was noted during the on-the-spot audits that the Commis-
30.
sion is not the only source of funding for these conferences and
in many cases other sponsorship has been received. However, no
steps were taken to ensure the total sponsorship/funding did not
exceed the total costs of the event.

(e) establish a structured interface between fellows and the world

of research, including industry;

On-the-spot controls by the Commission

(f) develop and foster the international dimension of Marie Curie

Fellowships;

(g) increase the appreciation and understanding of science by the

general public.

27. The Commission contribution to the MCFA, financed from
the TMR programme (see Table 4), represents the quasi-totality of
the resources of the Association.

Commission support to the Marie Curie Fellowship Association

Table 4

1998

1999

2000

Total

(EUR)

Commitments

445 442

280 000

—

725 442

Payments

311 810

89 088

296 544

697 442

Balance of unused commitment appropriations as at
31 December 2000

28 000

Source: European Commission.

28. The audit of the MCFA exposed a serious absence of man-
agement controls. Financial procedures concerning the eligibility
of expenditure were not in place, the effect of which was that all
expenditure incurred was deemed eligible for Commission fund-
ing. This culminated in a significant portion of the funding received
from the Commission not always being spent effectively.

Accompanying measures

29.
In the model contract for euroconferences, practical train-
ing courses and summer schools, contractors are asked to ensure
a balance of people under the age of 35, from less-developed
regions, to encourage female participation and to encourage indus-
trial participation. However, on the cost statements the contrac-
tors only have to prove that participants in the conferences are
under 35 years of age and consequently there was no evidence
that all the other criteria were systematically being checked.

31.
In the period 1996 to 1999 no on-the-spot financial controls
were carried out by the Commission, either by Commission offi-
cials or external contractors, on the TMR programme. It is impera-
tive that a minimum of checks be carried out early in a pro-
gramme to ensure both that the correct procedures are being
carried out by the contractor and that these procedures and the
objectives are realistic.

SURVEY RESULTS

32.
In the period September to December 2000 a total of 1 896
questionnaires were sent out to contractors participating in the
TMR programme. Questionnaires were sent, on a random sample
basis, to all categories of contractors/beneficiaries in all Member
States and all states associated with the framework programmes.
The number of contracts surveyed and their volume can be found
in Table 5. 1 372 (72,36 %) replies were received. An analysis of
the number of questionnaires sent and replies received can be
found in Table 6. The aim of these questionnaires was to evaluate
the performance of the Commission and the usefulness of the
TMR programme itself. The initial part of the questionnaires was
designed to obtain information on how involved the contractors/
beneficiaries were in earlier framework programmes and if they
were involved in other activities in 4FP. Each process in the con-
tract procedure from call for proposal, preparation of proposal,
evaluation, contract negotiation and signature through to report-
ing and payment procedures was questioned to determine how
satisfied the contractors/beneficiaries were. Finally, the contractors/
beneficiaries were asked if they would participate in the Fifth
Framework Programme.

Table 5

Summary by type and value of TMR programme projects audited
(on-the-spot/questionnaire)

Types of projects audited

Value
(in Mio EUR)

% of total
budget

Number of
contracts

Research networks

234,92

Access to large-scale facilities

91,80

Training through research
(Marie Curie Fellowships)

Accompanying measures

37,17

7,15

Total

371,04

Source: European Court of Auditors.

65,8

77,1

13,4

18,3

46,8

158

116

417

76

767

10.12.2001

EN

Official Journal of the European Communities

C 349/11

Table 6

Analysis of questionnaires sent and received

Total sent

Total
replies
received

Percentage

365

215

576
609

131

269

188

352
456

107

73,70

87,44

61,11
74,88

81,68

Research networks

Large scale facilities

Training through research (Marie
Curie Fellowships)
– Fellows
– Supervisors

Accompanying measures

Source: European Court of Auditors.

34. A point worth noting from the results concerns the access
to large-scale facilities activity. More than half of those surveyed
had previously held contracts in this area under the earlier frame-
work programmes and almost all had already submitted a pro-
posal in the Fifth Framework Programme. It is also interesting to
note that more than half the respondents in this area were also
involved in other activities in the TMR programme.

35. From the results it can be noted that the beneficiaries are on
average satisfied with the service they receive from the Commis-
sion. The area where the least satisfaction is expressed concerns
the calls for proposal and the selection procedures (44 % to 65 %).

33. The main results of this survey can be found in Table 7.
Many respondents also attached remarks to their returned ques-
tionnaires which provided further insights on the functioning of
the Commission and the TMR programme.

36. The area where beneficiaries are most satisfied relates to the
answers they received from Commission staff in response to their
queries (62 % to 77 %).

Results of the survey conducted on the spot and by post

Table 7

Percentage of beneficiaries

Research
networks

Large scale
facilities

Training
through research
(Marie Curie
Fellowships)
fellows

Training
through research
(Marie Curie
Fellowships)
supervisors

Accompanying
measures

1

2

3

4

5

6

7

8

9

10

11

Who previously held contracts in this area under the
Second and Third Framework Programmes

45 %

55 %

Who previously submitted applications for Marie
Curie Fellowships

—

—

Who have submitted a proposal in this area under
the Fifth Framework Programme

43 %

85 %

Who intend to submit an application for a further
research fellowship

Who are involved in other activities of the TMR Pro-
gramme

Who have income tax deducted from their stipend

Who are satisfied with the procedures in place for
calls for proposals, contract selection

Who are satisfied with the procedures in place for
contract negotiation

Who are satisfied with the procedures in place for
producing scientific reports and making payments

Who had all queries raised with the Commission ser-
vices quickly and satisfactorily answered

Who encountered no administrative or scientific prob-
lems during the Marie Curie Fellowship

—

28 %

—

58 %

59 %

57 %

62 %

—

—

51 %

—

60 %

62 %

68 %

77 %

—

17 %

—

47 %

—

79 %

44 %

—

—

—

72 %

—

—

—

65 %

—

76 %

70 %

—

—

47 %

—

50 %

—

29 %

—

60 %

68 %

64 %

77 %

—

Source: European Court of Auditors.

—

56 %

79 %

C 349/12

EN

Official Journal of the European Communities

10.12.2001

CONCLUSIONS AND RECOMMENDATIONS

Conclusions

Selection of proposals

37. The transparency of the procedures for selecting proposals
was not always evident:

(a)

in the allocation of proposals to evaluators, potential conflicts
of interest may arise; and

(b) there was an absence of detailed records of the deliberations

leading to the panel decisions made.

Contracts and management control

38.
In the training through research activity (Marie Curie Fellow-
ships) the Commission has taken significant steps in improving
the management of the programme.

39. The wide divergences in success rates for proposals in the
area of Marie Curie Fellowships between the TMR programme
and the specific thematic programmes is unsatisfactory and may
adversely affect the reputation of the fellowship scheme.

40. The department managing Marie Curie Fellowships in the
TMR programme shows how a centralised unit can operate effec-
tively and efficiently within the Commission.

Recommendations

Selection of proposals

41. The following procedures, related to the situations described
in paragraph 37, are proposed in order to improve the system of
evaluation.

(a) All applications from individuals to become evaluators of
proposals should be assessed and a decision taken on the suit-
ability of the applicant for the work. The applicant should be
informed of this decision. The experts remaining on the list
should be used for the evaluations. The subsequent selection
should be based on an analysis of the risk of potential con-
flicts of interest, covering areas such as home institutes, mem-
bership of associations and, to some extent, nationality.

have a defined length of time to evaluate each proposal. More
care should be taken in the allocation of proposals to ensure
potential conflicts of interest do not arise and that evaluators
have the necessary skills and qualifications for the proposals
they are assessing. A more thorough review of the results of
the evaluations should be carried out by the Commission to
ensure that proposals are properly ranked.

(c) Finally, to ensure transparency, full minutes should be taken
of all meetings and the discussions behind all decisions should
be produced.

Contracts and management control

42. Procedures should be put in place to ensure participants in
research networks are adhering to the eligibility criteria. As the
main problem lies in the recruitment of researchers who have
already received fellowships from the Commission, it should be
relatively simple to crosscheck researchers recruited on networks
against the existing database of past and present fellows in the
participating organisation. In addition, the Commission should
give more guidance on how to recruit researchers for the projects
and should also take steps to improve the user and friendliness of
the Cordis website.

43. The Commission is recommended to require contractors in
the access to large scale facilities activity to keep supporting docu-
mentation for all costs incurred. For travel and subsistence costs
a declaration should be obtained from the user that he/she has
received no other financial support.

44. While the Commission is encouraged to use the agreed user
fees approach for access to large scale facilities contracts with con-
tractors, it should ensure that the user fees are correctly estab-
lished. The advice of financial officers should be sought and fully
documented before contracts are signed and follow-up audits
should take place to ensure the user fees are correct.

45. The Commission should carry out its mid-term reviews in
line with actual contract duration.

46. The Commission should continue the management of Marie
Curie Fellowships in the department responsible for this activity
in the TMR programme. It should also make it responsible for the
management of the other fellowships in the specific programmes.

(b) No evaluator should be allowed to attend an evaluation where
his/her organisation has submitted a proposal. All evaluations
should be done in one building and the evaluators should

47. The Commission is encouraged to continue to offer good ser-
vice in answering contractors’ queries.

10.12.2001

EN

Official Journal of the European Communities

C 349/13

48. Should the Commission decide to further fund the Marie
Curie Fellowship Association,
it should ensure that adequate
management controls have been put in place, and it should pro-
vide the Association with strong guidance on how to spend the
money and also fix limits for expenditure in certain areas.

is also recommended that the Commission request, along with
the cost statements, declarations from the contractor showing the
total amount sponsored and the percentage of the total costs of
the event this sponsorship has covered. This could also be used
as an additional check on the accuracy of the costs declared.

49. The Commission should ensure that if target criteria are
included, particularly in the area of euroconferences, practical
training courses and summer schools, documentation is provided
by the contractor to ensure these criteria have been respected. It

50. The Commission should take steps to ensure on-the-spot
audits are carried out as soon as possible on contractors in the
Improving Human Potential (IHP) programme.

This report was adopted by the Court of Auditors in Luxembourg at its meeting of 19 July 2001.

For the Court of Auditors

Jan O. KARLSSON

President

C 349/14

EN

Official Journal of the European Communities

10.12.2001

THE COMMISSION’S REPLIES

SUMMARY

II. Generally speaking, the procedures for evaluating the fourth framework programme gave ample guarantee
of transparency. The presence of an independent observer also provided an extra guarantee of proper applica-
tion of the procedures.

In the cases mentioned by the Court, it is true that the minutes did not always fully reflect the different stages
leading to the decisions of the full panel. This had no impact on the results nor on the financing of projects
(see points 10 and 37).

In the fifth framework programme the whole evaluation procedure was improved by the use of a single evalu-
ation manual adopted by the Commission and made available to all potential participants. Moreover, an open
call for experts and a common expert database for all programmes have been set up. Finally, the status of
independent observer has been codified and for some activities the panel’s reporting system has been improved
with the designation of a rapporteur responsible for drafting the reasoning behind the decision (see points 10
to 12).

In an area like research, where there is often a concentration of experts at the forefront of science, the potential
risk of a conflict of interests cannot be excluded from the outset. Having said that, the two-stage evaluation
structure and in particular the peer review of each proposal, individually to begin with and then by the panel
as a whole, guarantees a balance between the potential risk of a conflict of interests and the quality of evalu-
ations. This procedure has been improved considerably under the fifth framework programme by steps such
as the coding of the status of independent expert and attaching terms of reference, a code of conduct and a
declaration of impartiality duly signed by the party in question to the contract of each evaluator (see points 11
and 37).

III. Checks are being made to clarify, together with the two organisations, the cases mentioned by the Court
of Auditors. Any measures needed will be taken (see points 17 and 42).

IV. The Commission services responsible for the negotiation of access to large-scale facilities contracts, includ-
ing staff from both the relevant scientific and administrative/financial units, take all the necessary steps to
ensure correct calculation of the unit cost, which in turn provides the basis for the user fees due by contrac-
tors. A number of access contract audits will be carried out to provide an assurance that the established system
of user fees is correct and sound (see points 21 and 44).

The Commission has carried out the TMR mid-term reviews considering the effective duration of access con-
tracts. The Commission considers technical reviews and the overall process of programme monitoring and
evaluation as extremely valuable for the improved management of contracts and of the programme as a whole
and has further strengthened this process in FP5 (see points 22 and 45).

V. Efforts have been made since the fourth framework programme to improve the system of grants (e.g. cre-
ation of a single entry point for the receipt of proposals). Harmonised evaluation procedures and criteria have
also been put in place in the fifth framework programme. In the Commission’s proposal for the new frame-
work programme all activities relating to mobility, including Marie Curie fellowships, have been grouped under
a single activity (see points 24 and 46).

10.12.2001

EN

Official Journal of the European Communities

C 349/15

VI. The Commission systematically checks the eligibility of costs (age, nationality) before making any pay-
ments. Other criteria of note (geographic balance, male/female parity, participation of less developed countries,
representation of industry) are checked on the basis of replies to the questionnaires distributed to the partici-
pants (see points 29 and 30).

Furthermore, the Commission will consider how to incorporate checks on sponsorship in the elaboration of
the rules for the next framework programme (see point 49).

VII. The Commission was interested to see the results of the survey on the TMR programme undertaken by
the Court of Auditors among the beneficiaries/contractors. It notes the high proportion of responses and wel-
comes their generally positive view of all parts of the TMR programme (see points 33 to 36).

SELECTION OF PROPOSALS

tion of impartiality duly signed by the party in question to the
contract of each evaluator.

Audit approach and general results

Research networks

10. The two-stage procedure, as summarised by the Court of
Auditors, rightly shows that the first stage was in preparation of
the final evaluation by the full panel. As regards the minutes of
the panel’s decisions, the main point is that they correctly reflect
the results of those decisions, namely, the classification in order
of priority of the proposals that the panel would like the Com-
mission to finance. This has always been the case even if in the
cases examined by the Court of Auditors the minutes did not
always show the details of the discussions that led to that particu-
lar order of priority. This does not compromise the final result of
the evaluation.

11. The evaluation system under the fourth framework pro-
gramme, with its common basic structure for each activity, took
account, within a coherent legal framework, of the characteristics
of the different activities (large-scale facilities, research training
networks, Marie Curie fellowships, accompanying measures).

As said in point 10, the Commission stresses that the panels’ dis-
cussions (plenary sessions) have always been minuted and that
they reflect the final decisions of the panels strictly, even if in
some cases the discussions leading to the decisions are not always
fully documented.

In a programme covering all areas of science and requiring a peer
review of projects, an expert may be asked to evaluate proposals
in which his organisation is participating without there being any
direct conflict of interests, especially where a big organisation is
concerned (e.g. national research institutes). Where there is a
potential conflict of interests, the mechanism of a two-stage
evaluation procedure provides a good guarantee, particularly as
this system has been improved in the fifth framework programme
by steps such as the coding of the status of independent expert
and attaching terms of reference, a code of conduct and a declara-

12. Particular attention has been attached in the fifth framework
programme to the reporting of the discussions of the panel and a
rapporteur has been appointed to provide consistency between
the observations, scores and final classification adopted by the
panel.

Access to large scale facilities

It is true that individual evaluators have sometimes ranked
13.
proposals without following precisely the guidelines given in the
specific evaluation manual for large-scale facilities. Nevertheless,
the final panel rankings obtained at the second meeting follow
scrupulously such guidelines (e.g. that the top ranking is given
only if all five criteria are marked as ‘high’). The Commission did
in fact overview the evaluations to ensure the correspondence
between the final marks and the actual ranking finally given by
the panel. The evaluation procedures under FP5, on the other
hand, are now simpler in this respect, in that primary evaluators
are not required to rank proposals but only to mark them, the
ranking being left entirely to the panel discussion.

Contract extensions

14. When selecting proposals from the third call under the
fourth framework programme, preference was given to the method
of using an amendment rather than a new contract for infrastruc-
ture support projects that already had a contract from the first
call for proposals. This amendment stated that it did not finance

C 349/16

EN

Official Journal of the European Communities

10.12.2001

projects already supported under the initial contract but rather
complementary activities (Article 3). Consequently, the period of
overlapping should not have posed any problems since there was
no conflict between the activities being funded. On the issue of
the use of funds, Article 3 also stated: ‘... no funds may be trans-
ferred between the additional contribution and the contribution
referred to in Article 3(1) of the contract’.

Last but not least, it should also be pointed out that several other
criteria are taken into account in order to minimise the risk of
potential conflicts of interest. One of them is to avoid evaluators
of the same nationality as the coordinator of a proposal. Of course,
in a multi-disciplinary panel such as this, evaluators must also be
chosen among those who have the necessary expertise. It is ulti-
mately up to the panel, in its collegiality, to verify and discuss
those cases where potential ‘indirect links’ may be suspected.

It is true however that a few contractors have expressed some
confusion in relation to charging costs for the original contract
and for the amendment in the same annual report and that pay-
ments concerning the original and additional financing could not
be clearly differentiated. This is the reason why the system, despite
its intention to facilitate a more continuous technical implemen-
tation of access contracts, has been abandoned in FP5.

The Commission agrees that ‘regular users’ represent a case of
direct link and has already brought such potential conflicts of
interest to the attention of recent evaluation panels (see the latest
report of the independent observer for the infrastructures action
of FP5, available on the Internet).

15. As the specific evaluation manual for large scale facilities
(LSF) clearly states: ‘... the chairman will ask the programme staff
to provide factual information (...) concerning the implementa-
tion of contracts for access with any facility already funded under
this or earlier programmes that have submitted proposals to the
current round (...), including the results from [earlier] technical
audits’. In other words, knowledge of previous proposals, and of
how they have performed, is considered beneficial to a better
understanding of proposals. In any case, about one third of all
expert evaluators must change from one evaluation round to the
next. This standard rule has always been followed to guarantee a
proper renewal of expertise in the panel.

The issue of the links between applicants and evaluators is very
complex. The specific evaluation manual for the LSF clearly defines
cases of ‘direct link’ between an evaluator and an applicant. Such
potential conflicts of interest are thus carefully avoided Some of
the 11 cases quoted by the Court, however, do not fall under this
category but are considered as ‘indirect links’. Nevertheless, poten-
tial cases of ‘indirect link’ are also very carefully examined, but
demand a case-by-case analysis. In particular:

— the case of evaluators acting as referees in a user selection
panel is not normally felt to constitute such a link. By defini-
tion, in fact, the members of a user selection panel are asked
to act as independent experts (as in all ‘peer review systems’),

— the case of users may be rather difficult to assess. While a
‘regular’ user, including someone who intends to become a
user, is certainly to be excluded, users who have used a facil-
ity in the past but who have no plans to continue using it,
may be considered, in fact, as the best judges of that facility.

CONTRACTS AND MANAGEMENT CONTROL

Research networks

17. The Commission has taken note of the Court’s audit results
and the two concerned institutions are being contacted in order
to obtain full information on the eligibility of researchers financed
under the TMR contract. The Commission will then reimburse
costs only for those researchers that have been appointed in
accordance with the terms of the TMR network contract, and
where appropriate will make any necessary adjustments.

18. The scarcity of students in many scientific fields and the
practical obstacles to transnational mobility, are two among the
many factors that makes it difficult for the networks to recruit
young researchers. Such issues, which are of wider significance,
are addressed in a communication adopted by the Commission
on 20 June 2001. This communication also addresses the prob-
lem of the dissemination of information on vacancies for research
positions within Europe (proposal of an Internet portal, provid-
ing easy access to the relevant websites). Furthermore it is intended
to launch a ‘raising awareness’ campaign on the opportunities
offered by the research training networks and other activities of
the programme, especially towards the associated countries.

There is a general need for regular interaction between contrac-
tors and the Commission on various aspects of the implementa-
tion of the projects. Increased attention will be paid to the early
warning of network contractors on questions relating to the
appointment of young researchers, emphasising that the publica-
tion of posts on the Commission’s website is only one of various
promotional measures to be considered. Continued attention will
also be paid to the improvement and user-friendliness of the Cor-
dis website.

10.12.2001

EN

Official Journal of the European Communities

C 349/17

Access to large-scale facilities

lowship proposals; however, there were no further coordination
measures implemented.

20. The Commission agrees on the potential risk that travel and
subsistence costs may be incorrectly charged and that appropri-
ate warnings will be given so that contractors should verify
whether such costs have not already been paid by the home insti-
tutions of the users. Such considerations will be taken into account
in the current development of the new funding instruments for
the next framework programme.

21. The adoption of ‘user fees’ requires the negotiation of a ‘unit
cost’, the calculation of which rests on information provided by
contractors during negotiation. This calculation is based on the
annual direct operating costs of the facility, based on historic
costs, and on the total amount of access provided over the same
period. The Commission services responsible for the negotiation
from both the scientific and
of contracts,
administrative/financial units, take all necessary steps to ensure
that such calculations are correctly established.

including staff

As the system of user fees was new in TMR, the negotiation pro-
cedures and the corresponding documentation have been much
more carefully described in FP5 in order to improve the general
understanding of this accountancy basis.

22. Large contracts extended (in some cases through ad-hoc
modifications to the contract duration) until the start of FP5 (i.e.
early 2000). Taking into account this overall time perspective the
timing of the mid-term reviews was appropriate. Moreover, oper-
ating as a dedicated mid-term review panel allowed the expert
reviewers to take a global view of the programme (including all
contracts), which is extremely valuable for the improved manage-
ment of the contracts and of the programme as a whole. This
could only be achieved by conducting the review after the final
call of the programme. For this reason, the new ‘transnational
access’ contracts of FP5 specify a ‘technical verification of the
project’ (Article 22 of the new standard contract) in order to allow
a more flexible and efficient implementation of the monitoring
process.

In the fifth framework programme, fellowships were coordinated
and harmonised under the responsibility of the improving human
potential programme and, in particular, a common set of evalu-
ation procedures, including minimum threshold for quality of
proposals, was defined. As, however, budgets available within the
thematic programmes varied substantially, success rates of the
proposals submitted were in some cases also quite different within
the limit of the respect of the quality of proposals.

To further coordinate and harmonise Marie Curie Fellowships in
the future, the Commission has proposed, for the next framework
programme, to operate all Marie Curie Fellowships within one
activity line of one programme.

Taxation of Marie Curie Fellows

25. The Commission is aware of the problem of the differences
in taxation applied to the fellows’ income. In the beginning of
FP4, the Commission set up a working group on this issue where
all the Member States and associated States were represented.
In the conclusions of this working group there was strong oppo-
sition from three Member States regarding a Community legisla-
tive initiative for an equivalent legal EC status for EC fellows.
Within the context of the work on the European research area, a
High-Level Expert Group on improving the mobility of research-
ers was set up by the Commission with representatives of the
Members States. The High-Level Group has identified obstacles to
mobility of researchers and has also noted taxation in this con-
text. Many of the taxation differences arise from differences in sta-
tus of the fellows in the Member States. Taxation is a difficult issue
to address at Community level, as decision-making relies on una-
nimity and for some Member States taxation policy lies at the
very heart of national sovereignty. This reluctance to discuss taxa-
tion issues has also been clearly seen in the course of the work of
the High-Level Group.

Training through research (Marie Curie Fellowships)

Follow-up of previous observations

These items are also addressed by the Commission in its forth-
coming communication ‘A mobility strategy for the European
research area’ (COM(2001) 331 final, of 20 June 2001).

23. The Commission is pleased to note the Court’s observation.

Marie Curie Fellowship Association

Decentralisation of selection and management of Marie
Curie Fellowships

24. The Commission is aware of the need for coordination and
harmonisation of Marie Curie Fellowships. In the fourth frame-
work programme, there was a unique entry point for all such fel-

28. The final financial and scientific reports of the Marie Curie
Fellowship Association (MCFA) have been submitted to the Com-
mission, who is evaluating them. On this basis, all necessary mea-
sures will be taken. The Commission has already announced to
the MCFA that should there be any future funding, improved
management control and financial rules will have to be put in
place as an a priori condition.

C 349/18

EN

Official Journal of the European Communities

10.12.2001

Accompanying measures

nature, the LSF action addresses facilities that are unique or rare
and that provide essential services to the users.

29. The Commission systematically checks the eligibility of costs
(age, nationality) before making any payments. Other criteria of
note (geographic balance, male/female parity, participation of
less-developed countries, representation of industry) are checked
on the basis of replies to the questionnaires distributed to the par-
ticipants.

35. The Commission welcomes the observation of the Court
concerning the services they provide to their contractors, in par-
ticular on the general satisfaction concerning the procedures for
calls and selections.

30.
It is true that there are cases among the accompanying mea-
sures where the Commission is not the only source of funding. In
such cases, the check asked for by the Court, namely whether
total sponsorship did not exceed total costs, could indeed not
have been done systematically, at the level of the regular cost
statements. For the successor activity under FP5, the Commission
has made sure that the cost statements include a requirement to
report on total costs and the contracts exclude any profit for the
contractor. The Commission has taken note of the Court’s state-
ment and will take it into account when elaborating the rules for
the next framework programme.

On-the-spot controls by the Commission

31. The Commission accepts that early checks are imperative.
For this reason subsequent to the period of 1996 to 1999 con-
sidered by the Court, the Commission has intensified its audit
efforts and has in recent years increased the number of audits car-
ried out on a random basis.

In the case of the TMR programme, 30 audits are in the process
or have been in the process in the 4FP. Among these 30 audits:

— since 1999, 13 audits have been closed,

— five audits are ongoing,

— 12 are scheduled for start in 2001.

SURVEY RESULTS

33. The Commission was interested to see the replies by con-
tractors to the two questionnaires sent by the Court of Auditors
on the TMR programme. It notes the high proportion of responses
and welcomes their generally positive view of all parts of the TMR
programme.

36. The Commission services endeavour to provide a fast and
effective assistance to all queries from contractors and from the
research community at large. This service is facilitated by the use
of a dedicated website on the Internet, which is constantly kept
up-to-date.

CONCLUSIONS AND RECOMMENDATIONS

Conclusions

Selection of proposals

In an area like research, where there is often a concentration
37.
of experts at the forefront of science, the potential risk of a con-
flict of interests cannot be excluded from the outset. Having said
that, the two-stage evaluation structure and in particular the peer
review of each proposal, individually to begin with and then by
the panel as a whole, guarantee a balance between the potential
risk of a conflict of interests and the quality of evaluations.

This procedure has been improved considerably under the fifth
framework programme by steps such as the coding of the status
of independent expert and attaching terms of reference, a code of
conduct and a declaration of impartiality duly signed by the party
in question to the contract of each evaluator.

The fact that the minutes do not give details of the discussions
leading to the classification by order of priority of the proposals
that the panel wishes the Commission to finance does not com-
promise the final result of the evaluation and is not synonymous
with a lack of transparency in the respective funding. See also
response to point 41 (a), (b) and (c).

Contracts and management control

34. The frequent involvement of facilities supported for access
over consecutive programmes and also in related RTD projects
and cooperation networks is due to the fact that, by its own very

39. Any variations in the success rates of the various pro-
grammes stem from the differences in funding made available in
the specific programmes themselves.

10.12.2001

EN

Official Journal of the European Communities

C 349/19

Under the next framework programme the Commission has pro-
posed that all Marie Curie Fellowships should come under a single
activity.

ments suitable for feedback for the proposal coordinator and
agree on an overall evaluation summary report, which is
signed by them. In the activity ‘networks and access’ a rap-
porteur appointed by the panel is now responsible for draft-
ing the reasoning behind the decision.

Recommendations

Selection of proposals

41.

(a) While agreeing with the auditors’ comments that the conflict
of interest is an important consideration in the selection of
evaluators, it is by no means the only criterion. The Commis-
sion services have to examine the experts’ details (CVs) care-
fully to ensure that only the most appropriate people, with
the necessary experience and competencies, and other char-
acteristics (academia/industry, gender, nationality, linguistic
capabilities, etc.) are used. The open call for experts for the-
evaluation of proposals under the fifth framework programme
and the common expert database for all programmes have
already improved the openness and transparency of the pro-
cess.

(b) The evaluation manual of the fifth framework programme
contains specific steps for avoiding any potential conflict of
interest. This is achieved by way of the contract concluded
with the expert and the attached terms of reference, code of
conduct and declaration of impartiality duly signed by the
party in question.
Under the fifth framework programme, for most programmes
and calls, experts are invited to a central location to carry out
the evaluation. However, where time permits and where issues
of commercial sensitivity are not a limiting factor, proposals
may be sent to experts, either as electronic or paper copies,
for them to carry out an initial evaluation at their place of
work. If the latter alternative is used, it is made clear in any
public supporting documentation for the particular call. For
the future, the Commission is examining the possibility of
using remote electronic proposal evaluation as used by a
number of national authorities. Evidently any remote evalua-
tion is subject to considerations of commercial confidential-
ity and the nature of the proposals being evaluated.

(c) Since the fifth framework programme standard procedures
are used for the various decision levels. Experts examine the
proposals assigned to them individually, filling in an indi-
vidual evaluation sheet with their comments and marks. Once
all the experts to which proposals have been assigned have
completed their individual assessments for a given phase, a
panel is brought together to discuss the marks awarded with
a Commission official acting as moderator for the group. Dur-
ing this consensus meeting, the experts attempt to agree on a
final mark for each of the groups of criteria and an overall
mark for the proposal. They justify their marks with com-

Contracts and management control

42. Already, under the TMR network activity the Commission
has taken different actions to ensure that network participants are
well-informed on the financial/administrative management of the
network project. During the contract negotiation phase all net-
work coordinators were invited to Brussels in order to discuss the
implementation of the network contract and its financial aspects.
In particular, special attention was given in these seminars to the
discussion of the eligibility criteria for young researchers.

The Commission has also provided the network partners with
‘Guidelines for the preparation of cost statements’. These guide-
lines serve as a quick reference on the basic contractual stipula-
tions concerning eligible costs and summarise in particular all rel-
evant aspects concerning the appointment of young researchers.
The Commission has put all relevant documents on the Internet
(http://www.cordis.lu/fp5/management/home.html) and will fur-
ther improve the user-friendliness of the Cordis website.

43. The provisions concerning supporting documentation given
in the model contract for access to large-scale facilities under TMR
have been further developed in FP5, for example to include ‘records
of the presence at the infrastructure of all users within the user
groups benefiting under the contract, their nationality, their place
of work and the access provided to them’. Contractors will be
asked to pay special attention to potential problems concerning
travel and subsistence costs. Such considerations will be taken
into account in the current development of the new funding
instruments for the next framework programme.

44. The Commission has confirmed the system of user fees,
which rests on the negotiation of a ‘unit cost’ to be derived from
the annual operating costs of the facility and the corresponding
amount of ‘access’ available, provides an efficient method of work.
This system has been continued in FP5 and should be maintained
in the next framework programme. The Commission services
responsible for the negotiation of contracts, including staff from
both the scientific and administrative/financial units, take all nec-
essary steps to ensure that such costs are correctly established. It
is foreseen to carry out a number of ‘access’ contract audits to
provide an assurance that the established system of user fees is
correct and sound.

45. The Commission has carried out the TMR mid-term reviews
considering the effective duration of access contracts. The Com-
mission considers technical reviews and the overall process of

C 349/20

EN

Official Journal of the European Communities

10.12.2001

programme monitoring and evaluation as extremely valuable for
the improved management of contracts and of the programme as
a whole and has further strengthened this process in FP5.

48. The Commission agrees with the Court recommendation if
the Association decides to ask for further financing.

In the project for the next framework programme, the Com-
46.
mission has proposed to manage all Marie Curie Fellowships
within one activity line under the responsibility of one director-
ate.

49. Since the fifth framework programmes these checks are
made systematically.

47. The Commission services endeavour to continue their tradi-
tion of providing reliable and effective services to their contrac-
tors and to the scientific community in general, including prompt
and thorough replies to the queries received from applicants, con-
tractors, users, and the scientific community at large.

50.
In a general way, the Commission has intensified its audit
efforts and has in recent years increased the number of audits car-
ried out on a random basis. The IHP programme, after the TMR
programme (see point 31), is included in the general audit effort
like any other RTD programme.