EN

Brussels, 4.9.2008

COM(2008) 533 final

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Ex-post evaluation of the ‘Information Society Technologies (IST)’ Thematic Priority in the Sixth Framework Programme for Research, Technological Development and Demonstration (RTD)

COMMUNICATION FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT, THE COUNCIL, THE EUROPEAN ECONOMIC AND SOCIAL COMMITTEE AND THE COMMITTEE OF THE REGIONS

Ex-post evaluation of the ‘Information Society Technologies (IST)’ Thematic Priority in the Sixth Framework Programme for Research, Technological development and Demonstration (RTD)

(Text with EEA relevance)

1. Introduction

This Communication highlights the findings and recommendations of an evaluation of the ‘Information Society Technologies (IST)’ Thematic Priority in the Sixth Framework Programme for Research, Technological Development and Demonstration (RTD), in the context of the i2010 [1] Strategy. It sets out the Commission’s initial reactions and the measures already taken or planned. The evaluation, carried out from May 2007 to May 2008, responds to the requirements of the Financial Regulation [2], its Implementing Rules [3], and the provisions for evaluation for the Seventh Framework Programme [4].

2. Background

The objectives of the Sixth Framework Programme [5] (FP6) were to strengthen the European Research Area and the scientific and technological bases of European industry and encourage its international competitiveness, and to promote research activities in support of other EU policies.

In FP6, implemented from 2003 until 2006, the major investment was in a number of thematic priorities intended to focus and integrate Community research: €3 984 million was invested in the ‘IST Thematic Priority’ and an additional €216 million was invested in high-speed networks to connect research institutions in Europe with others in the world. The budget for the ‘Information and Communications Technologies (ICT) Theme under the Seventh Framework Programme (FP7) is €9 050 million for the period 2007-2013.

3.

Conduct of the evaluation

The objective of the evaluation was to assess the systemic effects of the IST research activities under the FP6, and the extent to which they have contributed to the FP6 and wider EU strategic objectives. It focused on the relevance, efficiency, effectiveness, utility and sustainability [6] of the investment.

The evaluation was carried out by a panel of high-level independent experts [7], chaired by Mr Esko Aho, supported by a team of professional evaluators [8] The Panel analysed the programme documentation and indicators of outputs and impacts. They took evidence from key players from the ICT sector and organisations carrying out IST research activities, as well as from senior industrialists, venture capitalists and academics. The emerging findings were discussed with members of the European Parliament’s ITRE committee and with Member State delegations on the IST programme committee (ICTC).

The evaluation report is in Annex 1 and has been disseminated widely, including through Europa [9], and presented to the ICTC and the ICT Advisory Group.

4. The Commission’s initial reactions, measures already taken or planned

The Commission welcomes the report and takes full note of the 23 Recommendations in the following box. It proposes to make them the subject of a broad debate on European policy on innovation in ICT.

Continuing efforts are made to simplify and reduce administrative burdens, within the constraints of sound financial management. In the FP7, a flat-rate for overheads is accepted for some participants, including SMEs, and average personnel costs can be used. Prior financial viability checks are only carried out for the project coordinators and participants seeking more than €0.5 million in funding. The previous requirement for financial guarantees is replaced by a guarantee fund, and audit certificates are only requested for cost claims of over €0.375 million. There is also now a unique participant registration system, such that each participating organisation need only register once. Costs are reimbursed for each financial statement, and reporting schedules can be specific to each project.

To increase flexibility, the project consortia can re-distribute budget to participants without informing the Commission; and each consortium has full management autonomy and can change participants according to the rules they define in their consortium agreement, with a simple notification to the Commission.

To attract more small, high-growth companies, the Community can now reimburse up to 75% of costs for SMEs, universities and public organisations, compared to 50% under FP6. Management costs are reimbursed at 100% without a cap, and the advance payment is usually 50% of the total Community contribution.

The 23 Recommendations of the ‘Aho Report’: It is recommended that efforts be made to continue to consolidate public-private partnerships of a more permanent nature, such as Joint Technology Initiatives (JTIs), under the Seventh Framework Programme. It is recommended to continue the effort to ensure that support for SMEs and for large firms is not ‘compartmentalised’ into different measures or tools. A platform should be created under the Seventh Framework Programme for new and high-growth companies to meet venture capital investors. Participation from outside Europe should be encouraged in all projects, from both developing and industrialised countries. The advisory system — e.g. the IST Advisory Group — should be Internationalised by including top scientists and engineers from around the world. The latest international developments and challenges should be reflected in the work programme. A more flexible approach may be needed to integrate new, interesting developments in the field faster. The research effort should focus on creating and sustaining world leadership where Europe already has a comparative advantage and where Europe has a new opportunity to take the lead. Europe should be selective and not attempt to become a world leader in every area. The eInfrastructures approach should be expanded to more application-oriented and user-oriented platforms in other sectors. Accounting control in JTIs should be carried out by Member States and participating companies, with a minimum of intervention at Community level. The Panel strongly recommends developing a more trust-based approach towards participants at all stages. The existence of a few unfortunate examples should not be allowed to stand in the way of innovation. Shorter proposals should be required with fewer details of work packages and a focus on the appropriateness of partnerships, in particular the inclusion of highly innovative participants. More complete and helpful feedback should be provided to proposers whose ideas are not funded. A new approach should be tested whereby proposals are not fully evaluated initially. All applications passing a few basic checks should be given a small amount of ‘seed funding’ for an exploratory phase. After this, exploratory projects with successful results would be selected for full project funding. Financing projects based on actual performance rather than promises and reputation could both reduce the initial paperwork and be a viable way of attracting innovative (small) companies that would not otherwise consider applying for Community funding. Consideration should be given to expanding the two-step evaluation procedure from the Open part of the ‘future and emerging technology’ area to other parts of the programme — prospective participants first provide a broad outline of their project idea, and only provide a more refined plan once they are selected. Reporting, which is time-consuming and may be untimely, should be optimised to allow the participants to report when there is something to report. It should be possible to refocus research on different priorities if this becomes necessary during implementation. Similarly, more flexibility should be allowed in the composition of partnerships during the project, including the possibility of changing partners if the research takes a direction that would benefit from new partners or the replacement of partners. The Panel recommends a more strategic use of standardisation to create new EU-wide markets. Standard-setting is needed as a tool for pulling through innovations and creating viable markets for new products and services. Welcoming the recent Commission Communication on pre-commercial procurement, the Panel recommends that new initiatives be taken to allow public authorities to procure the development of innovative goods and services. The European single market needs to be made more effective for business angels and venture capitalists, and European investment funds need to be more effectively utilised to pull through innovations from the Framework Programmes. When this cannot be left to market forces, a more strategic approach to standardisation at European level, focused on interoperability and development of standards where there is a well-documented need for coherent innovative services and European leadership, will be in the broader public interest. The interconnection of large regional and national eInfrastructures should be further developed. EU-wide platforms and infrastructures are needed in sectors such as eGovernment (especially procurement), eHealth (cross-border applications), logistics and transport. Framework RTD should be complemented by other measures, in particular public procurement at both national and European level. |

The Commission agrees that additional efforts should be made to increase the global reach of the programme in areas where Europe can take the lead. All projects are now open to participation from more than 150 countries with international cooperation agreements, under the same conditions as for the 38 associated and Member States. A new funding instrument is available for wider international collaboration. The Commission will explore options to further increase the global reach of ICT research.

The Commission agrees that the European innovation environment is critical to effective exploitation of the ICT research results, and it welcomes in particular the Panel’s recommendations on the need for systemic changes. It is essential that Europe is made more attractive to investment in research and innovation and that the right conditions are created for small, high-tech companies to flourish and grow. As a first step, the Commission will examine mechanisms to help new and high-growth companies to meet venture capital investors. Secondly, the Commission will explore options to speed up the process of developing standards and specifications for innovative products and services. Lastly, the Commission will explore options for more holistic innovation ‘eco-systems’ [10] that would reduce the fragmentation of public and private markets for innovative solutions; exploit synergies; and coordinate European efforts more systematically. This effort would build on the Lead Market Initiative [11]; Pre-Commercial Procurement [12], and take account of the lessons learnt from Joint Technology Initiatives, Joint National Research Programmes, and European Technology Platforms.

5. Conclusions

The Commission invites the European Parliament and the Council to take note of the findings and recommendations of the evaluation Panel, and to reflect on how best to take them forward. In particular, the Commission recognises that further reduction of red tape remains a significant challenge. It shares the Panel’s view of the need to develop a risk-tolerant implementation of the rules on research funding. It believes that the European Parliament and the Council also have a role to play in enabling such a trust-based environment, and invites them to take this into account in future discussions of the FP Rules of Participation and the Financial Regulation.

The Commission will consult on the findings and recommendations with the aim to increase flexibility, cut red tape and to respond to the call for a greater strategic impact from EU level research. To this end, the Commission is launching an on-line public consultation on broader and longer-term policy implications alongside the present evaluation report, the results of which will be summarised in a Communication on ICT research and innovation in early 2009.

The interim evaluation of ICT RTD in the FP7 will also allow the effectiveness of new measures to be assessed. It will be available in late 2009 as a basis for further reflection.

Annex 1

Key Findings and Recommendations from the Evaluation Report

Innovations in Information and Communication Technologies (ICT) are vital to Europe's competitiveness and economic welfare. They form the foundation for a knowledge- based economy: their development and use is crucial to sustaining growth and productivity. Investment in information society research and technology development has therefore rightly been a thematic priority in the 6th Framework Programme for Research and Technological Development (IST-FP6) – allocated €4B from 2003 to 2006.

This evaluation of the IST thematic priority has looked at how effective the research has been at creating new knowledge and innovation, as well as converting these innovations into economic growth and welfare through Europe's broader innovation systems.

The research investment has been well managed and has been effective in reaching its goals. However, improvements can be made in the flexibility and simplification of the funding mechanism, and in strengthening the global impact of European research in this area. More flexibility in programme administration can help attract more new high-growth companies. Most of the Panel’s recommendations regarding the management of the research could be implemented by the Commission within the duration of the 7th Framework Programme (in 2010 to 2013).

Other recommendations may require a more long-term effort and/or reach beyond the domain of the Framework Programmes at a more strategic and political level. The extent to which new knowledge and skills created in Europe are exploited in Europe depends on a broad portfolio of policies and measures which affect the innovation "eco-system". Systemic change is needed to remove barriers to innovation and promote stronger interactions between users, researchers and business - notably in regional innovation systems.

The evaluation has identified a number of opportunities to improve the environment for innovations from ICT research in the European Framework Programmes. Greater synergies are needed with venture capital investment; with regional innovation strategies and with public procurement. A more strategic, European level, approach is needed to standardisation, lead market development and the mobilisation of public-private partnerships, as in the Joint Technology Initiatives launched as a result of activities in the 6th Framework Programme.

1. Management of the European research investment

1.1. Utility and sustainability – the role of IST-FP6

Much of the research funded by the EU under IST-FP6 would not have been undertaken, or only undertaken in a much reduced form, without European support. IST-FP6 has therefore played a significant role in taking research activities to European and global levels. This is especially true when effectiveness requires critical mass, broad discussions and “new eyes” on a problem, or when research needs to be shared with an advanced group of experts in the field.

The achievements have high sustainability. Participants gain significant benefit from strengthened networks, new knowledge and skills. The networking effects are crucial, with many participants developing long-lasting collaboration. Some of these networks have become stable structures that form the core of wider collaborative environments, such as European Technology Platforms. As a result, JTIs have been introduced as a way of strengthening public-private partnerships in research at the European level. The new JTIs on nano-electronics and "embedded systems" are good examples of how the Framework Programme will have a positive long-term impact.

It is recommended that efforts are made to continue to consolidate public-private partnerships of a more permanent nature, such as JTIs, from the 7th Framework Programme.

1.2. Effectiveness – the impact of IST-FP6 research

Overall, the participation rate of small companies (SMEs) has been sustained at over 20%, which is well above the 15% target set by the European Parliament and the Council. However, the involvement of high-growth companies in the programme remains low. This raises questions about the degree to which the Framework Programme is attractive and accessible to high-growth companies, and the degree to which participation in it assists access to venture capital and to European and global markets.

Both small and large firms are required to create the optimum environment for SME growth, and financing mechanisms should be open to both. Both types of firms have their role in the innovation eco-system and each is important to the other. It is positive that the collaborative research in the Framework Programme enables small and large companies to work together in most projects. However, the smaller "targeted research" projects must not become a special instrument for SMEs and larger Integrated Projects must not be overly dominated by large enterprises.

It is recommended to continue the effort to ensure that support for SMEs and for large firms is not “compartmentalised” into different measures or tools.

Research networks created through participation in European projects have increased the effectiveness of knowledge transfer among organisations and the speed of diffusion of information, as well as the mobility of human resources. Changes in research partnerships as a consequence of participation in the Framework Programme are one of the areas where the impact of EU investment is most visible. Many participants report that the strong networks created during the Framework Programme will continue as the basis for their future research cooperation.

Much of the research is “far from the market” or basic research, and commercialisation of new products and services is not a direct objective. Little has been done recently to link participants to the venture capital community. A facility to promote such links was beginning to be developed in the period up to the year 2002, but largely vanished with the economic downturn immediately thereafter. It is time to consider reintroducing one or more instruments for this purpose.

A platform should be created within the scope of the 7th Framework Programme for new and high-growth companies to meet venture capital investors.

In section 2.1 further recommendations are made to strengthen this link and to stimulate subsequent market innovations.

International cooperation between the EU and China, India and Africa has been strengthened and 60% of the top-25 global innovation-leaders are involved in the programme. Integrated Projects have been effective in connecting European IST research to the world innovation system. However, international co-operation should also be used to bring the best science and technology of the world to European researchers, so that they can build upon it. It is therefore recommended to further globalise the Framework Programme, as elaborated below.

1.3. Globalising the Framework Programme

Europe cannot afford to be an RTD “Fortress". European researchers need to be partners with the best in the world, whether they are from inside or outside of Europe. There is a need to open up further to the world so that developing European research and innovation can draw upon the best minds and the best ideas, regardless of their origin. In addition, some technologies developed in Europe will need to be commercialised in other markets, in collaboration with non-European firms, to the eventual benefit of European companies and consumers.

If the best researchers from around the world participate in the Framework Programme, it will also become more attractive for the best European researchers. This is particularly relevant for highly innovative smaller high-growth firms which are not currently well represented in the programme. Giving them the opportunity to work with the world’s best researchers and innovators would increase their motivation to take part in the programme.

Although cooperation, in particular with Asia and Africa, has been strengthened, the Panel recommends that these efforts are taken even further in order to make the programme truly global:

Encourage participation from outside Europe in all projects. Participation from both developing and industrialised non-European countries should be encouraged.

Internationalise the advisory system – e.g. the IST Advisory Group - by including top scientists and engineers from around the world

Reflect the latest international developments and challenges in the work programme. A more flexible approach may be needed to integrate new, interesting developments in the field faster.

Focus the research effort on creating and sustaining world leadership where Europe already has a comparative advantage and where Europe has a new opportunity to take the lead. Europe should be selective and not attempt to become world leader in every area

1.4. Relevance and strategic impact – IST-FP6 and wider EU objectives

IST research investments in 2003-2006 have contributed to sustain and create European world leadership in some research areas, although not always translated into lead market capabilities in ICT and applications. IST-FP6 has reinforced market leadership in mobile communications and research leadership in high-speed networking, GRIDs, advanced robotics and audiovisual systems. World leadership has been achieved in the development and use of high-speed e-Infrastructures for science and research. In these areas, opportunities have been created for new entrants (notably SMEs) in addition to the continued participation of leading companies. Support to emerging technologies has created world leadership in quantum communications, nano-electronics and complex systems.

The successful development of high-speed electronic networks (eInfrastructures) has demonstrated the importance of European intervention in infrastructure development. These initiatives have created European world leadership.

The e-infrastructures approach should be expanded to more application-oriented and user-oriented platforms in other sectors.

The European Research Area has been strengthened, especially through Integrated Projects, Networks of Excellence, and the clustering of projects. European Technology Platforms have extended cooperation beyond the individual projects financed under the IST-FP6. They have fostered wider synergies with national and private-sector initiatives and have been an essential step towards new Joint Technology Initiatives (JTIs). These are an incentive for industry and Member States to increase their R&D funding. They provide a way of creating new partnerships between publicly and privately-funded organisations involved in research, focusing on areas where research and technological development can contribute to European competitiveness and improving the quality of life. They can therefore be seen as a pioneering approach in pooling public and private research efforts and they have become an integrated part of the 7th Framework Programme.

Both JTIs and "living labs" [13] have the potential to further strengthen the innovation process in other areas. They represent a more “systemic” approach by bringing together the supply side (enterprises and researchers) and the demand side (users and public authorities). JTIs and "living labs" could also be used as vehicles for targeted public procurement of innovation. However, the Panel is concerned that JTIs may be subject to overly-strict EU administrative controls which may inhibit innovation and participation of innovative companies, as has been the case for the Framework Programme.

The Panel recommends that accounting control in JTIs is carried out by Member States and participating companies, with a minimum of intervention at the Community level.

1.5. Efficiency – cutting red tape

It is important to pursue the goals of the research in a cost-effective manner. Further efforts are needed on simplification and reduction of administrative burdens of the Framework Programme, both for participants and for the Commission. They have not been significantly reduced in the 6th Framework Programme, because of the unfamiliarity with the new instruments and the large number of partners in Integrated Projects and Networks of Excellence. Some improvements have been made in the transition to the 7th Framework Programme, but there is a need for further simplification and increased flexibility. The Commission should explore options for further simplification.

1.6. Simplifying management

The measures taken by the Commission to ensure responsible use of public funds can often inhibit the programme from reaching its full potential, for example many of the most innovative companies – in particular high-growth SMEs – are discouraged from participating in the research programme because of the cumbersome procedures for both application and implementation.

(...PICT...)

The system of evaluation of proposals can discourage newcomers: feed-back on the quality of applications to unsuccessful proposers is reported by participants as poor compared with communication from private sector funding sources; a consensus-based evaluation process tends to result in projects offering incremental developments rather than radical innovations; and it is difficult to attract the best scientists from industry as evaluators.

While these are common problems in all public research funding, some were exacerbated in the 6th Framework Programme by the introduction of new instruments and increased overheads linked to the management of very large project consortia.

The Panel strongly recommends developing a more trust-based approach towards participants at all stages. The existence of a few unfortunate examples should not be allowed to stand in the way of innovation. Specific elements in the development of such an approach are detailed below.

Efforts should be made to both simplify and introduce flexibility in the three key phases of the project lifecycle – the application, the evaluation of proposals, and the management of funded projects:

At the application stage, it is recommended:

To require shorter proposals with fewer details of work packages and a focus on the appropriateness of partnerships, in particular the inclusion of highly innovative participants.

At the evaluation of proposals stage, it is recommended:

That more complete and helpful feedback is made available to proposers whose ideas are not funded.

To test a new approach whereby proposals are not fully evaluated initially. All applications passing a few basic checks should be given a small amount of "seed funding" for an exploratory phase. After this, exploratory projects with successful results would be selected for full project funding. Financing projects based on actual performance rather than promises and reputation could both reduce the initial paperwork and be a viable way of attracting innovative (small) companies which would not otherwise consider applying for Community funding.

To explore expanding the two-step evaluation procedure from the Open part of the "future and emerging technology" area to other parts of the programme - prospective participants first provide a broad outline of their project idea, and only provide a more refined plan once they are selected. This may increase the workload for the Commission in the early phases, and lengthen the evaluation process, but it will significantly reduce the burden on the research community of preparing proposals.

At the stage of project management, the main problem is the rigidity of carrying out the research according to a fixed schedule and with a partnership established in advance. It is therefore recommended:

To optimise reporting, which is time-consuming and may be untimely, and allow the participants to report when there is something to report.

To allow the refocusing of the research on different priorities if this becomes necessary during implementation.

Similarly, to allow more flexibility in the composition of partnerships during the project, including the possibility of changing partners if the research takes a direction which would benefit from new partners or replacement of partners.

2. Broader policy recommendations: Realising the full benefit in growth and welfare through the European innovation eco-system

ICT is an enabling, pervasive technology, whose benefit depends on a dynamic "eco-system" of RTD and innovation. Supply-side support alone to RTD, such as that of the Framework Programme, is not sufficient. It is also necessary to improve the linkage of RTD into the innovation “ecosystem” in Europe. The approach to innovations must be to simultaneously "push" through RTD and "pull" through market demand and public services, with efforts needed simultaneously at national/regional level and at the European level.

For these reasons, the Panel has devoted a significant part of their evaluation to those factors which determine the outcomes and impact of European research. Many of the Panel’s recommendations therefore relate to systemic measures which lie outside the Framework Programme management, but which must be associated with it. There are several areas in which an effort should be made, but the Panel has focused on two key issues:

– Developing new markets, and

– Improving European infrastructures and interoperability.

2.1. Developing new markets

Innovation needs new capital investment. However, venture capital is less available in Europe than in the USA. During most of the 6th Framework Programme, the availability of early stage (seed and start-up) venture capital in the EU15 was approximately half of that in the US. In addition, European public authorities do not fully utilise their considerable purchasing power to foster innovation through procurement of innovative services and technologies.

Although knowledge flows in both directions between the EU and the US, there is a net flow from Europe to the US. Thus, while enterprises often choose Europe as a location for R&D, commercialisation of their research results in many cases takes place in the US, a tendency further supported by the fact that the top 50 global companies in the ICT sector are predominantly American or Japanese.

To improve the “ecosystem” of R&D and innovation in Europe, the demand for innovative solutions and the financing of high-growth innovative enterprises must be increased.

Strengthening the demand for innovative solutions

To ensure that R&D results are converted into growth and benefit for society, there is a need for an accompanying multifaceted, demand-oriented effort.

The Panel recommends a more strategic use of standardisation to create new EU-wide markets. Standard-setting is needed as a tool for pulling through innovations and creating viable markets for new products and services.

The public sector has the potential to create demand for innovative goods and services. Public procurement currently accounts for approximately 17% of Europe’s GDP. Historically, it has been one of the strong drivers of innovation and take-up of new technologies. However, Europe is lagging behind the US, where public procurement plays a key role in innovation. Thus, there is potential for significantly grater value to be generated by the Framework Programme research through increased public procurement of innovation at regional, national and EU level. At the same time, it should be emphasized that research and procurement of innovation should not focus only on products, but also on the development of new, innovative services.

The Panel welcomes the recent Commission Communication on pre-commercial procurement, and recommends that new initiatives are taken to allow public authorities to procure the development of innovative goods and services.

Increasing the availability of financing

Europe lags behind the US in the availability of finance to support the growth of companies. Not enough venture capital is available in Europe and many technologies and services with commercial potential, developed by small, highly innovative European firms, attract US investment. The growth benefits of these firms are effectively relocated from Europe to the US. Europe needs to be competitive not only in technology development but also in the availability of capital for innovation and growth. We must secure European resources for these companies to grow. This will also secure a new base of innovative companies capable of participating in Framework Programmes. Business angels, seed capital and full-scale venture capital need to be available in order to have a well-functioning financial market for innovation and growth.

The European single market needs to be made more effective for business angels and venture capitalists, and European investment funds need to be more effectively utilised to pull through innovations from the Framework Programmes.

2.2. Improving European infrastructures, standards and interoperability

The development of cross-border infrastructure, interoperability and, in some sectors, standards, is one of the areas where true European added value can be achieved.

In some cases, the market takes care of the development of de facto standards through competition. However, national regulations and practices can constitute barriers to the development of European and global standards. This fragments markets and prevents European companies from reaping the full benefits of a single European market.

A more strategic approach to standardisation at the European level, when this cannot be left to market forces, focused on interoperability and development of standards where there is a well-documented need for coherent innovative services and European leadership, will be in the broader public interest.

The interconnection of large regional and national eInfrastructures should be further developed. EU-wide platforms and infrastructures are needed in sectors such as eGovernment (especially procurement), eHealth (cross-border applications), logistics and transport. Framework RTD should be complemented by other measures, in particular public procurement at both national and European level.

[1] Web reference: http://ec.europa.eu/information_society/eeurope/i2010/index_en.htm.

[2] Council Regulation No 1605/2002 of June 2002 on the Financial Regulation applicable to the general budget of the European Communities (OJ L248/1 – 16.9.2002).

[3] Commission Regulation No 2342/2002 of 23 December 2002 laying down detailed rules for the implementation of the Council Regulation on the Financial Regulation applicable to the general budget of the European Communities (OJ L357/1 31.12.2002).

[4] Decision No 1982/2006/EC of the European Parliament and the Council concerning the Seventh Framework Programme of the European Community for research, technological development and demonstration activities (2007-2013) (OJ L412/1 30.12.2006).

[5] Web reference: http://fp6.cordis.lu/fp6/home.cfm.

[6] Web reference: http://ec.europa.eu/budget/evaluation/Key_documents/regureq_en.htm.

[7] Panel: Mr E. Aho (Chairman) - former Prime Minister of Finland and President of the Finnish National Fund for Research and Development (Sitra), Mr M. Cosnard - Professor at the Polytechnic School of the University of Nice-Sophia Antipolis, Chairman and CEO of INRIA, Mr H-O. Henkel - Professor at the University of Mannheim and former CEO of IBM Europe, Middle East and Africa as well as former President of the Federation of German Industries, Mr L. Soete - Director of UNU-MERIT, Mrs N. Stame - Professor, Social Policy, University of Rome ‘La Sapienza’ and co-founder and first president of AIV, Mr P. Telička - former EU Commissioner and Senior Advisor, European Policy Centre.

[8] Ramboll Management, contracted under the framework contract for evaluations and evaluation-related services managed by the Budget Directorate-General of the European Commission.

[9] Web ref.: http://ec.europa.eu/dgs/information_society/evaluation/rtd/fp6_ist_expost/index_en.htm.

[10] The network of researchers and entrepreneurs, interacting with investors and markets.

[11] Communication on ‘A lead market initiative for Europe"’ COM(2007) 860.

[12] Communication on ‘Pre-commercial Procurement: Driving innovation to ensure sustainable high-quality public services in Europe’ COM(2007) 799.

[13] Living labs are city- level centres of innovation and experimentation, generally in mobile technologies and applications, bringing together users and developers to create and test ideas and products.

--------------------------------------------------