REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL

Interim Evaluation of the European Metrology Research Programme - EMRP

1. Context and Overview of the EMRP

By Decision of 16 September 2009 of the European Parliament and the Council the EU agreed to participate in a joint research programme called Eureopean Metrology Research Programme – EMRP[1], with a contribution of up to €200 million for the period 2009-2017, equivalent to the contributions of the 22 Participating States[2]. The above mentioned Decision (hereinafter the “EMRP Decision”) is based on Article 185 TFEU (previous Article 169 EC Treaty) which, in implementing the Multiannual Framework Programme, makes it possible to coordinate national research programmes, through a voluntary integration process between the Participating States, covering scientific, management and financial integration. The joint implementation of the national research programmes requires the establishment or existence of a dedicated implementation structure. The participating States had agreed to propose EURAMET e.V.[3] as dedicated implementation structure to implement the EMRP. The dedicated implementation structure should be the recipient of the Union financial contribution and should ensure the efficient execution of the EMRP.

The core activity of the EMRP consist of funding multi-partner trans-national projects addressing research, technological development, training and dissemination activities (EMRP projects). In view of the concentrated capacities in metrology, the core part of the EMRP projects shall be executed by National Metrology Institutes and Designated Institutes (namely, specialist institutes responsible for certain national standards and associated services that are not covered by the activities of the National Metrology Institutes) from the participating States. In order to increase and diversify capacities in metrology, the EMRP also funds several researcher grant schemes which complement the EMRP projects.

In the EMRP Decision the Parliament and the Council recalled that metrology is a cross-disciplinary scientific field which is a vital component of a modern knowledge-based society. Reliable and comparable measurement standards, and appropriate validated measuring and test methods underpin the processes of scientific advancement and technological innovation and thus have a significant impact on the economy and quality of life within Europe.

In its Communication on the Innovation Union Flagship Initiative[4], adopted in the context of the EU's growth strategy “Europe 2020”[5], the Commission underlined the importance of smart, sustainable and inclusive growth for our citizens with targets and actions at EU and national levels. Central to the EU's strategy are all efforts to turn the European Union into an Innovation Union.

The concept of partnering in European research and innovation to pool forces, achieve breakthroughs and tackle societal challenges was put forward recently in a Communication from the Commission[6] to Council and European Parliament. Council recognised in its conclusions[7], that partnering can facilitate the optimal use of resources and hence develop full use of Europe’s intellectual capital and contribute to the integration of ERA as well as reduce unnecessary duplication. Partnering brings together European and national players to build critical mass, joint visions and strategic agendas, making use of flexibility, scale and scope. In the same manner the EMRP has established a strong partnership betweeen the 22 Participating States starting in 2009.

The EMRP Decision requires an interim evaluation of the EMRP carried out by the Commission with the help of independent experts three years after the start of the EMRP. This evaluation shall cover progress towards the objectives as well as recommendations concerning the EMRP on the most appropriate ways to further enhance integration and the quality and efficiency of the implementation, including scientific, management and financial integration and whether the level of the financial contributions of the participating States is appropriate, given the potential demand from their various national research communities.

The Commission shall communicate to the European Parliament and the Council the conclusions thereof, accompanied by its own observations.

An expert panel supported the Commission in this evaluation with its findings and issued the interim evaluation report (hereinafter the "Report") which provides an in-depth analysis of EMRP concerning the progress towards the initial objectives as well as recommendations on the most appropriate ways to further enhance integration and the quality and efficiency of this initiative.

The present Report provides the Commission views on the main recommendations expressed by the experts in the Report[8].

2. Interim evaluation of the Independent expert group: Commission’s observations

The Group’s Report covered all relevant aspects as requested by the EMRP Decision and provided an important number of facts, comments and recommendations. The Commission considers the whole Report as being an integral part of the Interim evaluation process. The Commission will therefore highlight, in the following sub-paragraphs of this chapter, the most relevant findings/recommendations concerning the EMRP and express its view respectively.

2.1 Quality and efficiency of the implementation

In delegating the implementation of EMRP to a Designated Implementation Structure (DIS) the Commission has followed the provision of Article 185 TFEU and the rules applying to indirect centralised management in accordance with the Financial Regulation. EURAMET e.V. has been the selected DIS and its governance structure has proved to be efficient and of high quality for the implementation of the EMRP. Considering not only the operational strength of EMRP but also its scientific content the Panel noticed that increased visibility of EMRP beyond the metrology community would be beneficial for the wider stakeholder community and society in general.

The Commission welcomes the strong engagement and professional management of EURAMET e.V. and the EMRP Committee in the implemantion of the EMRP in its first three years since 2009 and encourages all relevant instance in relation to the EMRP implementation to continuously improve the quality of implementation and the visibility of the programme.

The Commission endorses the Panel recommendation to establish a key performance indicator (KPI) for time-to-contract and set targets for improvement in order to start all projects as soon as possible after their selection.

2.2 Financial contributions from the Participating States

(1) The EMRP Decision made the Union contribution depend on the formal commitment by each participating State to contribute its share of financing for the EMRP and the effective payment of the financial contribution to beneficiaries. The total national commitment of all 22 Participating States was fixed to at least EUR 200 million and included 10% in cash mainly to cover all running costs (up to 16 million €) of the management of EMRP and for the remainder contributing to the grant schemes. In addition EURAMET provided evidence of adequate financial guarantees by the participating States to the Commission, in accordance with Article 8 of the Decision. The amounts guaranteed by the participating States are calculated according to a pre-established fixed allocation key per Participating State. The guarantees have been constituted by means of guarantee or liability declarations by the participating States or by their national metrology institutes towards the Commission.

(2) To date all financial contributions from the Participating States have been honoured in full respect of the procedures set out for the implementation of the EMRP. The national contributions to selected projects have been made in full respect of all agreed procedures.

The Commission recognises that the Participating States hounoured their initial commitments as layed down in the EMRP Decision and agrees with the Panel that EMRP has achieve a high level of financial integration with its financing model.

2.3 Integration of national metrology research programmes

The EMRP initiative aims at aligning and integrating relevant national metrology research activities to establish a joint research programme featuring scientific, management and financial integration.

The level of scientific integration can be considered as a major achievement of the EMRP but also as a major benefit for the national metrology programmes involved. The ‘Grand Challenge’ approach has fostered widely inter-disciplinary cooperation between the national metrology research programmes. Even so third parties are to some extent involved in the EMRP projects, their involvement and influence on the programming seems sometimes limited and could be increased in areas of strong multi-disciplinary nature. Whilst it seems appropriate that the main focus of the EMRP should be on the development and strengthening of competences and capabilities of the national metrology or specialised dedicated institutes, the Panel believes that third party experts can add value to this specialised community. In addition the involvement of the wider research community could be strengthened further in certain areas which go beyond the typical metrology research aspects.

In agreeing jointly on potential research topics for each call for proposals and deciding on which topics will be selected for further consideration, the EMRP Committee is the instance which takes a joint strategic view of the priorities for metrology research across Europe. The Panel highlighted that the EMRP research topics have been well chosen, that the EMRP is globally recognized and can avoid very costly duplications. An estimated 50% of the national investment is now coordinated within the scope and topics of the EMRP and thus it is clear that considerable scientific integration of core national funding is being realised.

However, the difference in committed resources to the EMRP between the large and small contributors is considerable as it was agreed in the Decision. Consequently the scientific competence and the existing research infrastructures are largely divergent which makes a full scientific integration between all 22 Participating States sometimes difficult. A major barrier for the small and developing national metrology institutes is their limited capability to participate with expert staff in EMRP projects and their limited financial resources. The Panel is of the opinion that this creates a strong risk that the existing gap in competence and capabilities between the well established, big metrology institutes and the developing, small metrology institutes will become bigger instead of smaller.

The Commission shares the Panel's view that the capacity gap between the Participating Countries seems to increase in some cases and agrees with the Panel opinion that the Researcher Grant System could be used more effectively to increase further the possibilities for countries with limited metrology research capacities. The Commission fully endorses the Panel recommendation to explore the degree of flexibility that could be applied to the management of the mobility grants to overcome the relocation barrier.

The management integration has been achieved within EURAMET e.V., which has proved to be a professional organisation that is working well. Where needed, improvements have been and are still being made so the EMRP can be considered as effective in its implementation. The current management structure is regarded as both efficient and integrated.

Nevertheless, the Panel noticed a need to explore ways of reducing and simplifying unnecessary bureaucratic procedures. The administrative burden for coordinators and participants in projects is considered high and the execution of projects may be delayed by the difficulties related to the grant contracts. The Researcher Grant model contract has been improved in 2011 and covers as of 2012 only essential aspects of the grant. In spite of these improvements, the Panel reported that the opinion persists that the Researcher Grant system is not working properly and should be reconsidered.

Another concern expressed by several EMRP Committee members and by the Panel is the necessity to have sufficient and competent project coordinators, who start the projects as soon as possible after the selection decision. The Panel concluded that there seems to be a general lack of professional competence in the management of complex international research projects.

The Commission agrees therefore with the Panel to harmonise further the management procedures and encourages EURAMET e.V. to provide European research project management training for project coordinators and potential project partners.

The financial integration of the EMRP is at a high level and the Panel concluded that further financial integration would be difficult to achieve due to the block funding nature of the national institutes which correspond to the national commitments of the Participating States. The EMRP uses a "virtual common pot" system combined with an appropriate reserve funding mechanism which allows financing projects equal to a "real common pot" system. The EMRP governance and its high financial integration make sure that available funding per call always matches the need from the proposers' side. In this way the order of the ranking lists of the central and independent evaluations has been for all three calls fully respected without any exception.

2.4 Progress towards the objectives

To analyse the progress towards the objectives of the EMRP, the Panel returned to the ex-ante impact assessment[9] that outlined the general policy objectives, the specific objectives and the operational objectives of the EMRP.

With regard to the general policy objectives the Panel concludes that the EMRP has made substantial progress in relation to the free movement of knowledge within the ERA. Strong cooperation between the main research actors in the field has been generated in order to support the development of knowledge and growth to support Europe's competitiveness at global scale. The partial focus on the grand challenges related to energy, environment and health is enabling more interdisciplinary collaboration within the metrology community to address societal need. The Panel concludes that the EMRP contributes to the realisation of the European Research Area (ERA) by implementing a genuine “European Metrology Research Area” (MERA).

With regard to the specific objectives the Panel underlines the structuring effect in relation to financial integration which has taken place already in the first years of the EMRP. There is also another useful structuring effect observed, that is apparent through the elaborate annual process of setting priorities for the Joint Calls, which seems to be influencing national priorities in some countries. The structuring effect and de-fragmentation is clearly greater in those countries operating with a centralised metrology system. It is too early to assess these structuring effects on industry, society and the exploitation of new technologies but the Panel seems convinced that a level of critical mass is being achieved that would have been impossible without the EMRP.

The EMRP operational objectives cover issues related to the grand challenges, capacity building, open access to infrastructures, increasing collaboration with the wider scientific community, modernisation, mobility of young researchers, better coordination of international affairs, supporting regulation & standards and supporting industry and economic growth. According to the Decision the EMRP is expected to achieve these operational objectives in the following way:

a) Pooling excellence in metrology research - by creating competitive joint research projects (hereinafter referred to as "EMRP projects") marshalling capability of sufficient critical mass from the networks of national metrology institutes and designated institutes from the Participating States to tackle major metrology challenges faced at European level;

b) Openness of the system to best science - by increasing participation from the wider European researcher community through researcher grants;

c) Capacity building - by increasing the capability of the European metrology researcher community through researcher mobility grants targeting those EURAMET Member Countries with limited metrology research capability.

The Panel highlighted with great satisfaction that the pooling excellence in metrology research has been achieved within the European metrology community. EURAMET e.V. and the EMRP Committee should be congratulated on facilitating the scientific coordination according to the Panel. The Panel encouraged possible improvements in the selection process for Strategic Research Topics for each Call, which could be even more supportive to opening the EMRP to all relevant stakeholder needs. Such an approach may be more supportive to exploitation of results at industry and regulatory level and may call for an enlargement of the existing metrology research capabilities within the national institutes and beyond.

The Commission endorses the Panel recommendation and encourages EURAMET e. V. to explore the potential added value of organising stakeholder workshops to prioritise Strategic Research Topics, especially for Grand Challenge Calls where a more open-minded culture would be desirable.

The Commission further agrees with the Panel's recommendation that certain research topics would benefit from an increased weighting of the impact criteria related to rapid exploitation of results within Europe.

In contrast to the great success of pooling excellence in the core metrology community itself, the opening of the system to the best science has so far been limited. The Panel reports a clear external perception that the EMRP still seems to be rather closed to the wider European research community.

The Commission therfore encourages EURAMET e.V. as recommended by the Panel to explore ways to better use the grant schemes to foster links with the best centres of excellence across Europe.

In its further analysis the Panel finally concludes that the EMRP is not having the desired effect in terms of capacity building in those countries with limited or no metrology research capability. While some countries have taken advantage of the programme to build capacity in strategic areas of interest, the capacity gap with the most research-intensive countries seems to increase.

It should also be noted that the mechanisms in the existing EMRP related to opening of the system to the best science and the capacity building have been based mainly on the grant system within EMRP. It seems that the financial capacity of the grant system itself is not the major bottleneck. The Panel reminded that the existing EMRP is designed by the EMRP Decision as a research programme striving towards scientific excellence with its specific operational processes and financial instruments and the running EMRP can not easily support in addition the complex issue of capacity building. In this respect the Panel has given interesting and important recommendations for a potential longer term future of the EMRP.

The Commission shares the Panel views on the progress towards the objectives as reported and the Commission fully endorses the recommandation to use expert facilitators to foster better inclusion of those countries with limited metrology research capacity with the aim of closing the gap with the more advanced countries.

2.5 European added value of the EMRP

The Panel expressed its strong view that the EMRP is an excellent model of what can be achieved by coordinating core-funded national R&D programmes. The Panel highlighted three particular aspects of the EMRP in providing substantial European added value:

· EMRP allows for critical mass to address even complex, interdisciplinary topics such as the grand societal challenges, which would be beyond the capacities of a single country.

· EMRP is pooling substantial resources and brings together research efforts in the field of metrology from 22 countries. The Panel estimates that about 50% of total dedicated metrology funding in Europe is now coordinated through the EMRP, which along with the EU contribution, amounts to a EUR 400 million in a single joint programme.

· The Panel recognises a "de-fragmentation" of research efforts and observed that a reduction in unnecessary duplication is achieved through both the elaborate planning and implementation of each joint call (joint programming).

Against this background the Panel encourages EURAMET e. V. to consider how EMRP project results can be rapidly exploited by European industry and suggests focusing more attention on innovation and knowledge transfer activities on a longer term basis.

Another domain that has both European and national value added is the EMRP in its support to regulation in general like for example environmental regulation such as the European Water Framework Directive. The Panel recalled that the development and definition of metrological aspects of such regulations are examples where the metrology research community collaborating under the auspices of EMRP and EURAMET e.V. should be playing a leading role in developing underpinning new measurement methods. The Panel is of the opinion that this requires a wide level of foresight activities and early engagement with policy makers and regulatory authorities at national and European level. Only if this level of cooperation is achieved the EMRP will be able to exploit its full potential in support of the Union policies.

The Commission encourages EURAMET e.V. on the basis of the Panel recommendation to explore options for foresight workshops with regulatory ministries/agencies and the relevant Commission Directorate-Generals.

In addition to possible foresight activities towards regulatory matters, the Panel noticed that metrology aspects in relation to ‘new technologies’ is a field in which the metrology community can support the growth of emerging sectors, where there is a need to develop new or more accurate measurement methodologies. "New technologies" seem therefore an area where there should be more collaboration with the wider research community either directly within EMRP projects but also potentially through coordination with complementary FP7 projects and actors.

The Commission welcomes the Panel recommendation to explore the potential added value of creating incentives to enable cross-fertilisation between complementary EMRP and FP7 actors and projects.

2.6 The future development of EMRP beyond FP7 and lessons learned for future joint programmes in general

The Panel is of the opinion that beside the possible above mentioned improvements for the running EMRP, there are also a number of lessons for future joint programmes in general. These lessons include some that are generic and others that are specific to the situation of the European metrology community – for example, the extensive use of national ‘institutional or block’ funding.

The Commission welcomes, without pre-empting any decision on the future of EMRP beyond FP7, the initiative taken by the Panel to advise on eleven possible issues to be considered for the design of any future initiative which should enable EURAMET e.V. to reach for higher levels of European integration.

Concerning the lessons for future joint programmes the Panel expressed its view that EMRP could be in some cases an example for Joint Programming activities or other initiatives, in case they would make use of Article 185 TFEU for the implementation of a joint programme. The Panel concluded that not all interested Member States may have yet suitable running programmes to integrate within a joint programme or the programmes are not flexible enough to do so. It needs to be considered that in most EU countries, limited competitive funding is available for coordination at European level.

In the opinion of the expert panel, it is particularly difficult to achieve financial integration under Article 185 initiatives and EMRP is a showcase how a "virtual common pot" based on in-kind contribution can achieve a very high level of financial integration.

The Commission welcomes the Panel reflections concerning future joint programmes and will continue its coordination activities between the different running Article 185 initiatives, in order to draw lessons learned for the future to provide for support to potential public-public partnerships under Horizon2020.

3. Conclusion

Having started in 2009, the EMRP operational performance has achieved maturity as a joint research prgramme between 22 Participating States, implemented by EURAMET e.V. The integration between the participating national programmes is considered high. The Commission will therefore continue to support the current programme as foreseen in the EMRP Decision.

The EMRP is performing well after 3 years to most of its initial operational objectives in what concerns nearly 85% of the EMRP resources, namely the pooling of excellence in metrology research. However there are significant gaps between expectation and reality in relation to three qualitative impact indicators: capacity building, interaction with the wider scientific community and mobility.

EURAMET e.V. and the EMRP Committee as the highest instances in the EMRP governance are invited to make all necessary efforts to improve this situation in the remaining period of the programme. While no changes to the initial Decision are considered necessary the above mentioned Panel recommendations should be implemented by EURAMET e.V. as well as any additional measure that EURAMET members may consider useful or necessary to improve the capacity building, the interaction with the wider scientific community and the mobility within the EMRP.

Finally, the Commission will during the next years be open to engage with EURAMET e.V. into preliminery discussions on the possible follow up for the current EMRP in the context of the next programming period, without prejudice to the final decision about HORIZON 2020 and the EU Multi-Annual Financial Framework, taking into consideration the wider political context of the EUROPE 2020 strategy.

[1] DECISION No 912/2009/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 16 September 2009 on the participation by the Community in a European metrology research and development programme undertaken by several Member States (OJ L257, 30.09.2009, p.12).

[2] Belgium, the Czech Republic, Denmark, Germany, Estonia, Spain, France, Italy, Hungary, the Netherlands, Austria, Poland, Portugal, Romania, Slovenia, Slovakia, Finland, Sweden and the United Kingdom, as well as Norway, Switzerland and Turkey

[3] Non-profit making association under German law

[4] Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions of 6.10.2010, “Europe 2020 Flagship Initiative Innovation Union” COM(2010)546 final

[5] Communication from the Commission of 3.3.2010, “Europe 2020” A strategy for smart, sustainable and inclusive growth' (COM(2010) 2020 final

[6] COM(2011) 572 final Partnering in Research and Innovation

[7] COUNCIL OF THE EUROPEAN UNION Brussels, 8 December 2011 18349/11 RECH

[8] http://ec.europa.eu/research/evaluations/index_en.cfm

[9] SEC(2008) 2949 Impact assessment report