# C_2008145EN.01000601.xml

| 11.6.2008 | EN | Official Journal of the European Union | C 145/6 |
| --- | --- | --- | --- |

11.6.2008

EN

Official Journal of the European Union

C 145/6

---

Council conclusions on the Special Report No 9/2007 by the European Court of Auditors concerning ‘Evaluating the EU Research and Technological Development (RTD) framework programmes — Could the Commission's approach be improved?’

(2008/C 145/03)

THE COUNCIL OF THE EUROPEAN UNION,

RECALLING the Council conclusions on improving the examination of special reports drawn up by the Court of Auditors in the context of the discharge procedure (1).

1

RECALLING that Article 163(1) of the Treaty provides that ‘the Community shall have the objective of strengthening the scientific and technological bases of Community industry and encouraging it to become more competitive at international level, while promoting all the research activities deemed necessary by virtue of other chapters of this Treaty’ and the important contribution of the Framework Programmes (hereinafter ‘FPs’) to achieving the goals of the Lisbon Strategy, including in terms of increasing human resources in research and development, giving high priority to public investments in science and research and fostering private investments, reforming public research infrastructures and promoting internationalisation.

RECALLING that previous evaluations of the FPs, such as the Five Year Assessment in 2004, provided an important contribution towards the launch and adoption of subsequent FPs.

RECALLING the current legal obligations of the Commission regarding the monitoring, evaluation and communication of information of FPs, in particular Article 7 of Decision No 1982/2006/EC of the European Parliament and of the Council of 18 December 2006 concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013) (hereinafter ‘FP7’) (2), Articles 27 and 28 of Regulation (EC) No 1906/2006 of the European Parliament and of the Council of 18 December 2006 laying down the rules for the participation of undertakings, research centres and universities in actions under the Seventh Framework Programme and for the dissemination of research results (2007-2013) (3), as well as the Council Decisions on Specific Programmes (4).

2

3

4

| 1. | WELCOMES Special Report No 9/2007 by the European Court of Auditors concerning ‘Evaluating the EU Research and Technological Development (RTD) framework programmes — Could the Commission's approach be improved?’ (5) (hereinafter ‘SR’); and RECOGNISES the efforts and achievements of the European Court of Auditors in producing this comprehensive SR. |
| --- | --- |

1.

WELCOMES Special Report No 9/2007 by the European Court of Auditors concerning ‘Evaluating the EU Research and Technological Development (RTD) framework programmes — Could the Commission's approach be improved?’ (5) (hereinafter ‘SR’); and RECOGNISES the efforts and achievements of the European Court of Auditors in producing this comprehensive SR.

5

| 2. | GIVES serious consideration to the overall conclusion of the SR and WELCOMES its Recommendations. |
| --- | --- |

2.

GIVES serious consideration to the overall conclusion of the SR and WELCOMES its Recommendations.

| 3. | TAKES NOTE of the Commission's replies which accompany the Report and WELCOMES the actions being already undertaken or envisaged to improve the Commission's approach to evaluations. |
| --- | --- |

3.

TAKES NOTE of the Commission's replies which accompany the Report and WELCOMES the actions being already undertaken or envisaged to improve the Commission's approach to evaluations.

| 4. | NOTES that the Court of Auditors' SR addresses the period covered by the Fourth to the Sixth EU Research and Technological Development FPs and that in addition to evaluation, the SR addresses a wider set of issues such as the structure, the coordination and the implementation of the FPs. |
| --- | --- |

4.

NOTES that the Court of Auditors' SR addresses the period covered by the Fourth to the Sixth EU Research and Technological Development FPs and that in addition to evaluation, the SR addresses a wider set of issues such as the structure, the coordination and the implementation of the FPs.

| 5. | RECOGNISES that the European Court of Auditors has observed that relevant Commission services implementing the FPs have complied with the formal Commission-wide requirements, including those specified in the 2000 and 2002 Communications on evaluation. |
| --- | --- |

5.

RECOGNISES that the European Court of Auditors has observed that relevant Commission services implementing the FPs have complied with the formal Commission-wide requirements, including those specified in the 2000 and 2002 Communications on evaluation.

| 6. | UNDERLINES, also in view of the forthcoming mid-term review process of FP7, that monitoring, evaluation and review play a critical role in the effective operation and possible adaptation of FPs (‘learning programmes’), and in helping to shape future policies for research both at European and national level. |
| --- | --- |

6.

UNDERLINES, also in view of the forthcoming mid-term review process of FP7, that monitoring, evaluation and review play a critical role in the effective operation and possible adaptation of FPs (‘learning programmes’), and in helping to shape future policies for research both at European and national level.

| 7. | NOTES that the Commission established an overall evaluation framework in 2000 which was revised in 2002 and 2007 and that relevant adjustments are being made continuously with particular reinforcement in the FP7 as set out in the FP7 ex ante Impact Assessment (6), in the Internal Control Standards (7) and in the recently developed ‘Quality Assessment Framework’ (8). These improvements include a new impetus to the Commission's existing internal ‘RTD evaluation network’, a better routine collection of data from projects, a new IT system for FP7, a more coherent and systematic use of performance indicators and the encouragement of scientometric approaches; and is therefore LOOKING FORWARD to the forthcoming ex post evaluation of FP6 in 2008. |
| --- | --- |

7.

NOTES that the Commission established an overall evaluation framework in 2000 which was revised in 2002 and 2007 and that relevant adjustments are being made continuously with particular reinforcement in the FP7 as set out in the FP7 ex ante Impact Assessment (6), in the Internal Control Standards (7) and in the recently developed ‘Quality Assessment Framework’ (8). These improvements include a new impetus to the Commission's existing internal ‘RTD evaluation network’, a better routine collection of data from projects, a new IT system for FP7, a more coherent and systematic use of performance indicators and the encouragement of scientometric approaches; and is therefore LOOKING FORWARD to the forthcoming ex post evaluation of FP6 in 2008.

ex ante

6

7

8

ex post

| 8. | STRESSES that, despite these adjustments, further efforts need to be undertaken to improve the research monitoring and evaluation systems on evidence based approach; therefore: |
| --- | --- |

8.

STRESSES that, despite these adjustments, further efforts need to be undertaken to improve the research monitoring and evaluation systems on evidence based approach; therefore:

| 9. | INVITES the Commission to pay utmost attention to the recommendations of the SR and in particular to:






  (a) | ensure greater clarity in the intervention logic in the conception and implementation of FPs in the context of relevant Community decision making procedures, in particular at programme and project level, with coherent and better defined scientific and socio-economic objectives which should be specific, measurable, achievable, relevant and timed, and balanced performance indicators, whilst taking into account the specific nature of RTD activities; |
| --- | --- | --- |

9.

INVITES the Commission to pay utmost attention to the recommendations of the SR and in particular to:

(a)

ensure greater clarity in the intervention logic in the conception and implementation of FPs in the context of relevant Community decision making procedures, in particular at programme and project level, with coherent and better defined scientific and socio-economic objectives which should be specific, measurable, achievable, relevant and timed, and balanced performance indicators, whilst taking into account the specific nature of RTD activities;

| (b) | establish a unified and coherent evaluation strategy across the relevant Commission services including the appropriate evaluation methodologies and techniques and, to this end, prepare a manual for the evaluation of FP activities, and in this context further develop the use of evaluation panels with the assistance of external experts with a view to ensuring the necessary level of independence; |
| --- | --- |

(b)

establish a unified and coherent evaluation strategy across the relevant Commission services including the appropriate evaluation methodologies and techniques and, to this end, prepare a manual for the evaluation of FP activities, and in this context further develop the use of evaluation panels with the assistance of external experts with a view to ensuring the necessary level of independence;

| (c) | consider options for a more centralised evaluation and reinforced coordination between the relevant Commission services, including the possibility of establishing a joint evaluation office; |
| --- | --- |

(c)

consider options for a more centralised evaluation and reinforced coordination between the relevant Commission services, including the possibility of establishing a joint evaluation office;

| (d) | optimise the collection, analysis and use of evaluation data in such a way to better measure the achievement of programme objectives in terms of outputs, outcomes and socio-economic impacts, whilst avoiding undue administrative burden for participants, and also maximising the usefulness of such data for decision making at European, national and regional level; |
| --- | --- |

(d)

optimise the collection, analysis and use of evaluation data in such a way to better measure the achievement of programme objectives in terms of outputs, outcomes and socio-economic impacts, whilst avoiding undue administrative burden for participants, and also maximising the usefulness of such data for decision making at European, national and regional level;

| (e) | establish the type and scope of evaluation in particular to give greater attention to assessing long-term impact of past and present FPs with a particular emphasis on European competitiveness; ensure timely and sufficient information on the results to stakeholders with a view to taking future policy decisions. |
| --- | --- |

(e)

establish the type and scope of evaluation in particular to give greater attention to assessing long-term impact of past and present FPs with a particular emphasis on European competitiveness; ensure timely and sufficient information on the results to stakeholders with a view to taking future policy decisions.

| 10. | INVITES the Commission to inform the Council and the European Parliament on the steps taken with regard to the recommendations of the Court of Auditors to improve the monitoring and evaluation of the FPs by end 2008, and to present the evaluation strategy for FP7 in a timely manner. |
| --- | --- |

10.

INVITES the Commission to inform the Council and the European Parliament on the steps taken with regard to the recommendations of the Court of Auditors to improve the monitoring and evaluation of the FPs by end 2008, and to present the evaluation strategy for FP7 in a timely manner.

| 11. | INVITES the European Court of Auditors to follow-up its audit in time for the 2010 discharge exercise and report back to the Council and the European Parliament. |
| --- | --- |

11.

INVITES the European Court of Auditors to follow-up its audit in time for the 2010 discharge exercise and report back to the Council and the European Parliament.

| 12. | INVITES the Member States and the Commission, taking into account the political priority given to governance in the European Research Area, to strengthen their cooperation and the exchange of approaches and expertise concerning the evaluation of the impact of the national participation in the FPs and the national research and innovation programmes, for example by the ‘European RTD evaluation network’, in order to promote more coherence among these impact studies of the Commission and the Member States. |
| --- | --- |

12.

INVITES the Member States and the Commission, taking into account the political priority given to governance in the European Research Area, to strengthen their cooperation and the exchange of approaches and expertise concerning the evaluation of the impact of the national participation in the FPs and the national research and innovation programmes, for example by the ‘European RTD evaluation network’, in order to promote more coherence among these impact studies of the Commission and the Member States.

---

(1)  Doc. 7515/00 FIN 127 of 3 April 2000 + COR 1 of 12 April 2000.

1

(2)  OJ L 412, 30.12.2006, p. 1.

2

(3)  OJ L 391, 30.12.2006, p. 1.

3

(4)  See the respective Annexes concerning the information to be provided by the Commission to the Committee (e.g. Annex V of Council Decision of 19 December 2006 concerning the specific programme ‘Cooperation’ implementing the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007 to 2013) (OJ L 54, 22.2.2007, p. 30)).

4

(5)  Doc. 5136/08 FIN 4 RECH 5 ATO 5 COMPET 5 of 9 January 2008.

5

(6)  Doc. SEC(2005) 430.

6

(7)  Doc. SEC(2007) 1341.

7

(8)  http://ec.europa.eu/budget/documents/evaluation_en.htm

8

---