EUROPEAN COMMISSION Brussels, 1.6.2016 SWD(2016) 126 final COMMISSION STAFF WORKING DOCUMENT Analysis of the implementation of the Regulation (EU) No 1025/2012 from 2013 to 2015 and factsheets Accompanying the document REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL on the implementation of the Regulation (EU) No 1025/2012 from 2013 to 2015 {COM(2016) 212 final}



Contents 1. Introduction 2. Background to the initiative 2.1 European Standardisation System (ESS) 2.1.1 Baseline for the intervention at Union level 2.1.2 The new ESS and its legal framework, regulation (EU) No 1025/2012 3. Evaluation Questions 4. Methodology 4.1 The Independent Review 4.2 "Article 24" Study 5. Implementation state of play - the results 5.1 Introduction 5.2 Implementation of the Regulation (EU) No 1025/2012 5.2.1 Transparency 5.2.2 Inclusiveness 5.2.4 Union financing of standardisation activities 5.3 Vademecum on European standardisation in support of Union legislation and policies, 6. Answers to the evaluation questions 6.1 Effectiveness

6.1.1 Objectives 6.1.2 Transparency 6.1.3 Addressing legal ambiguities 6.1.4 Use of ICT technical specifications in public procurement 6.1.5 Economic impact of standardisation: estimates from national studies 6.2 Conclusions regarding effectiveness 6.3 Efficiency 6.3.1 Efficiency of the European standardisation system (ESS) and Regulation 6.4 Conclusions regarding efficiency 6.5 Relevance 6.5.1 Relevance of the intervention 6.5.2 Relevance of the standardisation activities receiving Union financing 6.6 Conclusions regarding relevance 6.7 Coherence 6.8 Conclusions regarding coherence 6.9 EU added value 6.9.1 The added value of the ESS for stakeholders 6.9.2 The added value of the standardisation activities funded by the EU 6.10 Conclusions regarding EU added value 7. Overall conclusions

Annexes Annex 1: List of Acronyms Annex 2: The actors in the ESS Annex 3: Figure 1: The European Standardisation System (ESS) Annex 4: Three main problems identified in the Impact Assessment of 2011 Annex 5: Mandating Process, standards development and publication of the references in the OJEU Annex 6: Table 1 – The evaluation questions Annex 7: Article 24 Study and relationship with the Independent Review / Article 25 Report Annex 8: Methodologies and sources of information of the Independent Review Annex 9: Methodologies and sources of information Robustness of findings/limitations Annex 10: Table 4: Overview of the findings related to the implementation of the Regulation, by article Annex 11: Table 5: Number and percentage of NSBs/NSOs providing or planning to provide online public commenting and online access to draft national standards, 2013 and 2014 Annex 12: List of ICT technical specifications eligible for referencing in public procurement Annex 13: Table 10: Adopted standardisation requests in 2014 and 2015 Annex 14: Table 11: Notified draft requests in 2014 and 2015 Annex 15:Final Report of Independent Review of the European Standardisation System, Ernst&Young 2015 Annex 16:Final Report - Annexes of Independent Review of the European Standardisation System, Ernst&Young 2015 Annex 17:COM(2015)198 final REPORT FROM THE COMMISSION TO THE EUROPEAN PARLIAMENT AND THE COUNCIL Article 25 Report to the European Parliament and the Council on the impact of the procedures established by Article 10 of Regulation (EU) No 1025/2012 on European standardisation on the timeframe for issuing standardisation requests Annex 18:Study on the implementation of the Regulation (EU) No 1025/2012 (Article 24)

Annex 19:Report on the Friends of Standardisation Roundtable of 1st July 2015 Annex 20:Analysis of the results of the Online Survey of the Independent Review of the European standardisation system Annex 21: SWD(2015)205 final of 27 October 2015, Vademecum on European Standardisation in support of Union Legislation and policies. 1. Introduction The purpose of this evaluation is to assess the performance of the European Standardisation System (ESS – see Annex 1 for a list of acronyms) including Regulation (EU) No 1025/2012 on European standardisation (hereinafter “the Regulation”) since its entry into force in 2013, in terms of effectiveness, efficiency, coherence, relevance and EU value added. This exercise is conducted in response to the review clause set out in Art 24(3) of the Regulation. Given the recent entry into force of the Regulation, a full ex post evaluation of the Regulation cannot be achieved due to the fact that the full cycle of the standardisation work requested by the Commission since 2013 has not yet been completed. However, this exercise assessed whether the ESS is on track with achieving the objectives expected by the legislator in order to deliver, where appropriate, recommendations on how to improve the implementation of the Regulation and the system. It covers the EU and EFTA countries. It assesses mainly the processes put in place within the ESS and in the context of the Regulation and does not aim at assessing comprehensively the wider impacts of standardisation on the single market or the competitiveness of EU business. This evaluation is linked to the Regulatory Fitness and Performance Programme of the Commission because of the importance of standardisation as a way to harmonise the Single Market and of its impact on the competitiveness of the EU economy. This evaluation is of particular relevance to regulatory fitness because it draws upon studies or reports in relation to the ESS and its processes, specifically in relation to further involving particular stakeholders such as SMEs, refining Union financing and addressing the Administrative Burden on the European standardisation organisations (ESOs) and the National standardisation bodies (NSBs). This evaluation builds on two external studies carried out by the European Commission ("Independent Review", "Article 24" study - see section 4 for details) and on its Article 25 Report on the impact of the procedure established by Article 10 of the Regulation on the timeframe for issuing standardisation requests as well as on validation workshops held on 26 March and 1st July 2015. 2. Background to the initiative 2.1 European Standardisation System (ESS) European standardisation is a cornerstone for the functioning of the Single Market as well as a tool to support EU legislation and policies in technical harmonisation of product and service rules in the Single Market. There is no Single Market without standardisation. Due to its nature, standardisation concerns directly or indirectly every stakeholder, from industry to consumers to social and societal actors. Overall CEN and Cenelec have developed some 20,000 European standards (EN) or other deliverables and one (1) EN may in theory replace up to 33 national conflicting standards. This makes the business environment of European companies, notably SMEs, much easier. ETSI has developed some 35,000 technical specifications including some 450 harmonised standards. However, standardisation within EU/EFTA relies on voluntary cooperation between businesses, users, public authorities and other interested parties (consumers, NGOs and other stakeholders). At the EU/EFTA level, in simplified terms, the CEN/Cenelec standard setting process is based on consensus between the various industry players and other interested stakeholders participating in the standardisation process and where the final adoption of an EN is voted in a weighted vote between the NSBs. From a practical perspective, cooperation is needed between a broad range of stakeholders, including the EC and EFTA in case of the Commission requested standardisation, ESOs, industry, and the representatives of societal stakeholders and other stakeholders. Standards are voluntary technical specifications adopted by a recognised standardisation body for repeated or continuous application. European standards can be divided into 2 categories: 1. European standards adopted at the initiative of undertakings, the NSBs or other stakeholders. The majority (around 80%) of European standards fall into this category;

2. European standards supporting EU legislation or policies, developed at the request of the Commission, on the basis of a standardisation requests ('mandates') in which the ESOs are requested to draw up technical specifications of a normative nature that meet the requirements set out in the request. These standards can be subdivided into 2 subcategories: – Harmonised standards adopted by the ESOs for the application of Union harmonisation legislation for products or for services; – Other European standards in support of Union legislation or policies. The ESS involves a large number of actors who have different interests. At the heart of the ESS is the public-private partnership between the European Commission/EFTA on the one hand and the ESOs on the other hand. CEN, Cenelec and ETSI are recognised by Article 2(8) of the Regulation as the European standardisation organisations. Each of the ESOs is an independent, non-profit organisation that operates as a platform composed of two main parts: a “secretariat” and a “network”. Other major actors of the ESS are in particular the national standardisation bodies (NSBs), the international standardisation bodies, the Annex III stakeholders, and industrial stakeholders. The main actors of the ESS and their role are described in more detail in Annex 2. For the mutual interactions between these various actors see Figure 1 in Annex 3. 2.1.1 Baseline for the intervention at Union level In 2011, the Commission assessed the way the European standardisation system was functioning and three main problems (for details see Annex 4) were identified: 1.speed of Commission requested standardisation: the process for adopting European standards requested by the Commission was not fast enough (ie adoption took an average of 5 years in 2009) 2.under-representation of SMEs, social and societal stakeholders in European standardisation; SMEs were in general under-represented in standardisation activities, in particular at European level. During the public consultation on the reform of the ESS in 2010, 69% of respondents declared that the participation of SMEs in European standardisation should be reinforced. Furthermore, although standards play a major role in society, the opinion of relevant social and societal stakeholders was considered as not sufficiently integrated in the EU standardisation process 3.problems in using ICT standards developed outside ESOs and ISO/IEC in public procurement: in the field of ICT, many standards ensuring interoperability are not elaborated by the ESOs, but by global fora and consortia, however it was not possible to use these standards in public procurement. The co-existence of three different legal instruments on European standardisation i.e. Directive 98/34/EC (adoption of standardisation requests, transparency of standardisation work programmes), Decision 1673/2006/EC (financing ESOs) and Decision 87/95/EEC (ICT standardisation) created a number of ambiguities and did not authorize the Commission to adopt standardisation requests on service standards or standards supporting Union’s policies. There was therefore a need to remove uncertainties in the existing legal framework. 2.1.2 The new ESS and its legal framework, regulation (EU) No 1025/2012

In June 2011, following the conclusion of the Impact Assessment (IA), an overall revision of the system was initiated with the launch of the 'Standardisation Package'. The Standardisation Package is made up of the Commission Communication for standards [COM(2011)311], while the legal framework is set by Regulation (EU) No 1025/2012 that entered into force in 2013. The “Standardisation Package” answered the need to simplify and adapt the legal framework governing European standardisation, in order to reflect the latest developments and future challenges. The Standardisation Package also responded to the call of the European Parliament (EP) on the Resolution "Future of European standardisation" of 2010 in order to make the ESS fit for the future. The three major issues identified above (speed, inclusiveness, lack of or insufficient European standardisation in the field of ICT), identified by the preparatory work, were targeted by the reform. The Communication COM(2011)311 established the guiding principles of the European Standardisation system (ESS), expressed through five strategic objectives:

European standards need to be quickly available; European Standards are powerful strategic tools for businesses to increase their competitiveness; European standards need to respond to an increasing demand as a tool to support European policies and legislation; European standards will affect more and more groups in European society. The European standardisation system must, therefore, become as inclusive as possible; European standards must play a role in supporting the competitiveness of European business in the global market. As part of the reform, the Regulation sets the legal framework for European standardisation. In particular, the Regulation represents the consolidated legal basis for the Union to use European standards for products and services in support of the Union’s legislation and policies, to identify ICT technical specifications and to finance European standardisation. Entered into force on 1st January 2013, it aims at strengthening the role of European standardisation as a policy tool to support Union legislation and policies, not only for products but also for services. The concept of standardisation and the use of European voluntary standards in the service sectors are explicitly encouraged in the Directive on Services in the Internal Market (2006/123/EC) which is recalled in the Regulation. The main general objectives of the Regulation are to increase the contribution of standards and European standardisation to a better functioning internal market, stimulating growth and innovation and fostering the competitiveness of EU enterprises, especially SMEs. The Regulation's main specific objectives were to: increase the transparency of the standards development in the ESS; reduce the time taken by the Commission requested standardisation process; ensure that SMEs and societal stakeholders are adequately represented in the standardisation process, especially for standards developed at the request of the Commission; enable Member States to reference ICT technical specifications developed outside the ESS in public procurement; and set revised rules for financing European standardisation including SME and societal stakeholder organisations at the European level. The Regulation introduces several changes affecting the processes and the interactions among the actors involved. In particular, the following provisions should be noted: An improved planning, through the Annual Union Work Programme (AUWP) for standardisation, in order to identify priorities for European standardisation and the standardisation requests; Substantial changes to the mandating process (see Annex 5) are introduced: -After notification of the request, the (relevant) ESO(s) need(s) to communicate its (their) acceptance within one month (Art. 10(3) of the Regulation). -Conditional acceptance is not appropriate anymore — considering the formal and regulated process leading to adoption of a request — and should be considered as a rejection of the request by the ESO(s). Member States consultation was performed on the basis of Regulation (EU) 98/34. -The Regulation introduced the Committee on Standards (Art. 22 of the Regulation), being a comitology committee as defined in Regulation (EU) 182/2011 and supporting the Commission in implementing Regulation (EU) 1025/2012. As comitology procedures only apply to Implementing Acts, the legal form of standardisation requests was changed to Commission Implementing Decisions.

-Article 12(b) of the Regulation introduces and formalises required notifications to stakeholders about requests. Notifications need to be sent during the mandating process and before the standardisation request is adopted. -The introduction of a maximum time for the acceptance of the request by the ESOs and the fact that conditional acceptance is considered as a refusal increase the need for early informal consultations and discussions as ESOs need to start planning their resources (human, financial) and discussion, before the final and approved version of the standardisation request. Annex 5 illustrates the mandating process, standards development and publication of the references in the OJEU. 3. Evaluation Questions The present evaluation investigates the functioning of the European standardisation system (ESS) from two different perspectives: i) the overall ESS and the framework established by the new standardisation package and the actions undertaken by the different stakeholders and ii) the standardisation activities supported by Union financing. Each of the issues was investigated via a set of questions related to the five evaluation criteria (effectiveness, efficiency, relevance, coherence, and EU added value). Table 1 in Annex 6 recaps the evaluation questions for each of the issues, and links them to the relevant evaluation criteria. The answers to the evaluation questions are provided in section 6. 4. Methodology This evaluation relies on two external studies, the "Independent Review of the European Standardisation System (ESS)", an externally commissioned study prepared under Article 24(3) examining the implementation of the Regulation, and the Commission report on Article 25 of the Regulation. Annex 7 illustrates the relationship between these three documents. 4.1 The Independent Review The Independent Review of the ESS was carried out by an external contractor between December 2013 and March 2015 and involved an extensive consultation with all stakeholders. The main sources of information for the Independent Review consisted of a desk research, phone and face-to-face interviews with stakeholders, an online survey assessing in a quantitative and qualitative way the perception of stakeholders regarding the achievement of the strategic objectives of ESS, the efficiency of the processes and suitability of the governance, as well as the suitability of ESS for the future, and 10 case studies focusing on specific topics and aimed at providing an in-depth analysis of some critical points and areas for improvement identified in the course of the Independent Review. The methodology used in the Independent Review also included validation workshops which were organised (with the support of the EC) in Brussels on 26 March 2015 before the completion and publication of the final report of the Independent Review and a roundtable with stakeholders which was held after the publication of the final report on 1st July 2015. The Independent Review triggered an unprecedented response from stakeholders. More than 120 phone and face-to-face interviews were performed, with a vast range of different stakeholders. The online survey collected 447 complete answers, in addition to 348 incomplete answers (for a total of around 800 inputs). The first validation workshop attracted a broad stakeholders' participation as well, with over 110 participants (and 200 in July) including the Commission, ESOs and industrial, social and societal stakeholders. All these inputs provided a significant information basis for the Independent Review. For more details on the methodology and sources of information of the Independent Review see Annex 8. 4.2 "Article 24" Study In June 2015, the Commission tasked an external consultant to perform a study aiming at analysing the implementation of the Regulation, evaluating the relevance of the standardisation activities receiving Union financing and assessing possibilities for simplifying the financing of the European standardisation and reducing the administrative burden. The findings of the Article 24 study are based on a robust methodological approach which included both primary, qualitative sources (interviews with the relevant stakeholders) and secondary, qualitative/quantitative sources (for example, the ESOs 2013/2014 reports and the Commission’s data on Union financing). However, specific objectives were set for the Article 24 study. It was, therefore, outside of the scope of the study to examine the wider context and impact of standardisation activities on society and European businesses.

For more details on the methodology and sources of information of the Article 24 study see Annex 9. 5. Implementation state of play - the results 5.1 Introduction This section presents an analysis of the implementation of the Regulation, based on the information provided by the ESOs’ and the Annex III organisations’ reports. Table 4 in Annex 10 summarises the progress achieved in relation to each article of the Regulation (column entitled ‘Summary of the implementation’), the possible issues recorded in the implementation, and gaps in the information provided (column entitled, ‘Remarks and gaps in information’). 5.2 Implementation of the Regulation (EU) No 1025/2012



After two years of implementation of the Regulation it should be noted that: The Regulation has introduced the AUWP, which enhances the convergence of the individual plans of the various standardisation actors (EC, ESOs, NSBs etc.); Progress has been made in ensuring the transparency of the work programmes of the ESOs and NSBs, through their publication on the related websites; The transparency of standards has been enhanced. The ESOs and NSBs are gradually implementing tools to ensure access to draft national standards to all relevant parties; Significant investments in ICT tools have been made in the last number of years by the three ESOs and their members, in order to streamline and speed up processes; At European level, four stakeholder organisations have been selected as eligible for Union financing (SBS in 2013; ANEC, ECOS and ETUC in 2014) and these have launched activities to strengthen their presence in and contribution to European standardisation; Together with the new Framework Partnership Agreements between the EC and the ESOs, which include Key Performance Indicators (KPIs) and, among these, 50% reduction in the standard development time by 2020, the introduction of the AUWP and the revision of the procedure to adopt standardisation requests are expected to contribute to speeding up the standards setting process; In application of the Regulation, in 2015, the Commission has authorised the use of lump sums for the reimbursement of costs of the certain standardisation activities carried out by the ESOs and their members; At the national level, the NSBs report the participation, at policy and technical level, of representatives of four stakeholder groups: SMEs, consumers, social and societal stakeholders. Union financing of activities of the ESOs and the Annex III organisations is limited to a range of between EUR 20 to 25 million annually. The Commission contributes, on the basis of the Regulation, on average between 35% and 40% of the total income of the three non-profit ESOs and 75% to 100% of the total income of the Annex III organisations. The other sources of income are membership fees or equivalent other contributions and fees for organisations in partnership with the ESOs. At national level, NSBs usually enjoy public funding as an essential income element and in many Member States the membership fees of the ESOs are 'earmarked' from the national budget for the NSBs meaning that the Member States also indirectly support the operation of the secretariats of the ESOs (the legal entities but not the 'network'). Like the ESOs, the NSBs also collect membership and participation fees. For CEN and Cenelec related NSBs, revenues from sales of standards are an essential source of income together with other possible services (like training, certification, technical assistance, consultation) linked to application of standards. The experts who voluntary participate in standardisation work at the international, European and national levels, usually do so at their own expense. According to an estimation of the ESOs, the industry experts alone spend around EUR 1 billion each year to participate in and contribute to voluntary standardisation work (ie the estimated total value of the working hours, travelling and accommodation costs etc…). In addition, the industry buys standards and other related services from the NSBs. For the last 10 years, the ESOs and NSBs have implemented many online tools to distribute standardisation documents and other information to enable online meetings during standards development. Despite this fast development, no figures are available on how much the industry reduces its costs by effective use of these tools (ie less lost working time, less travelling and accommodation costs, no specific hardware or software are needed because of web applications).

Although the annual public funding of the legal entities operating in the ESS is almost negligible compared to overall costs borne by industry, in its current setting this public funding is crucial to keep the ESS running and essential for the ESOs to accept the Commission’s standardisation requests in support of Union legislation and policies. It provides a unique opportunity to include Annex III organisations and to ensure transparency, which is important for SMEs in their innovation strategies and for acceptance of standards. The application of the Regulation is recent. It has introduced several changes in the functioning of the overall system, not limited to Commission requested standards and standardisation deliverables, and which are being implemented by the Commission and applied or “enjoyed” by different parties involved (the ESOs, the NSBs the Member States and the Annex III organisations). Standards development processes and related activities now take 3 years. Moreover, in the case of new technical domains even longer and all the benefits produced can only be appraised in a longer time span. Therefore, the full assessment of the results achieved by the implementation of the Regulation and the activities supported within its framework will be performed at the next review. Nevertheless, the present evaluation will assess whether the implementation of the Regulation is on track to achieve what was expected by the legislator. According to the Article 24 study, no major issue is recorded in the application of the Regulation by the key stakeholders involved (see Table 4 in Annex 10 for details). 5.2.1 Transparency Progress has been made in ensuring the transparency of work programmes of the ESOs and NSBs, through their publication on the related websites. Some improvements as indicated in Table 4 in Annex 10 are however still needed. The transparency of standards appears ensured overall, thanks to the provision of information about European standards and the status of the work on the ESOs websites and their information systems. NSBs are gradually implementing tools to ensure access to draft national standards for all relevant parties (in particular those established in other Member States) and provide the opportunity to submit comments. The most direct and effective method is considered to be via online public commenting and online access to draft national standards. Although in 2013, less than 50% of NSBs were providing online platforms, this share is gradually increasing. In 2014, more than 84% of ETSI NSBs and 77% of CEN NSBs enabled online public commenting and online access to draft national standards. Cenelec NSBs are still lagging behind, with a share of 58% (though this percentage is an improvement on 2013). Several NSBs are planning to implement online platforms in 2016. The development of online commenting platforms is also supported by the ‘eComment project’, co-funded by the Commission, which will further develop the ESOs' national members’ capacity to offer draft standards for on-line public commenting via their websites. When online platforms are not available, access to national standards during public enquiry is usually provided via email. Further details regarding the number and percentage of NSBs/NSOs providing or planning to provide online public commenting and online access to draft national standards can be found in Table 5 in Annex 11. 5.2.2 Inclusiveness A range of provisions of the Regulation supports the participation of relevant stakeholders’ groups in standardisation, at the European level (through the Annex III organisations), and at the national level (through NSBs). At the European level, the Annex III organisations participate in the work of the three ESOs at policy and technical level in different ways, depending on the different models of CEN and Cenelec, and ETSI. At the national level, NSBs report the participation of representatives of SMEs, consumers, environmental stakeholders and societal stakeholders at policy and technical level. Also, several measures favour the access of SMEs to the standardisation process and to European standards and standardisation deliverables (e.g. reduced national membership fees or reduced prices for standards). Between 2013 and 2014, the number of ESOs’ technical bodies and NSBs recording stakeholder participation has been increasing. Despite this, data suggests that SMEs are largely, but not fully, represented in the relevant technical bodies at NSB level . The other stakeholders’ groups, and especially social and societal stakeholders, take part only in a small share of technical bodies at NSB level. However, the reporting on stakeholders’ participation at the national level would benefit from a clearer description of the measures taken to encourage participation, the forms of participation foreseen, and how the representation takes place. In parallel, in 2013 and 2014, the Annex III organisations have set up their activities to actively participate in and contribute to European standardisation, with concrete actions undertaken for the provision of experts participating in technical bodies of the ESOs, training and information and awareness raising. 5.2.3 Timing No complete assessment can be done for the implementation of the provisions related to Commission’s standardisation requests to the ESOs since 2013 as full development cycles of the requested standards have not been realised yet. This is also reflected in the Article 25 Report on the impact of the procedure established by Article 10 of the Regulation on the timeframe for issuing standardisation requests which concludes that 'the short timeframe to gain experience with the new processes does not allow to draw a final conclusion about whether the new rules will result in a permanent and unacceptable increase in the length of procedures for”[planning, preparing and adopting]“ standardisation requests'. Due to the transition from the previous regime under Directive 98/34/EC to the current legislation and due to the time needed to prepare standardisation requests, three standardisation requests were adopted in 2014 by the Commission under the Regulation and all were accepted by CEN within one month. In 2015, the number of adopted standardisation requests increased, with eight requests adopted by end of October 2015. The number of draft requests notified in the notification system was 12 draft requests in 2014 and 14 draft requests by end of October 2015. Thirty-two possible requests are identified in AUWP for 2016: historically, the trend was around twenty requests addressed annually to the ESOs, however this also included requests for studies and programming work not leading to publication of any standards or other specifications. This effect is due to the setting-up of the infrastructure resulting from the Regulation (ie Comitology, AUWP, the revision of the Vademecum on European standardisation). More detailed analysis concerning the impact of the Regulation on the planning, preparation, consultation and adoption of the requests will be available only after some years following new standardisation requests and the full completion of standards development processed by the ESOs. 5.2.4 Union financing of standardisation activities The list of the operating grants and action grants was signed by the Commission (DG GROWTH) with the ESOs and Annex III organisations. From 2010 to June 2015, 62 action grants related to the revision or development of standards and to preliminary or ancillary work were signed. Of these, 32 grants were signed in 2013-2014. The subject matter of the action grants allows us to gain an overview on the kinds of activities funded, the sectors and areas of work involved.

According to Article 15 of the Regulation, the Commission may provide funding to the ESOs, NSBs and other bodies contributing to the following standardisation activities: Development or revision of standards suitable for the support of Union legislation and policies, Art. 15(1) (a); Verification of quality and conformity to Union legislation and policies, Art. 15(1) (b); Preliminary or ancillary work, Art. 15(1) (c); Activities of the central secretariats of the ESO, Art. 15(1) (d); Translation of European standards or European standardisation deliverables, Art. 15(1) (e); Activities to explain, interpret or simplify European Standards (e.g. Information, user-guides, trainings), Art. 15(1) (f); Technical assistance, cooperation with third countries, promotion of the ESS, Art. 15(1) (g). One of the instruments used to simplify the financing of standardisation activities and to reduce the administrative burden is the reimbursement of lump sums by the Commission in case of activities under Art 15(1)(a) and (b). The Commission with the decisions taken in 2014 and 2015 authorised the use of lump sums for the reimbursement of costs of these standardisation activities carried out by the ESOs and their members. This simplification should enable the ESOs to better plan the Commission requested work and reduce the time for preparing a proposal for a grant to be financed by the Commission. At the same time, Art. 16 of the Regulation provides for the possibility to finance the Annex III organisations, in relation to their activities relating to European and international standardisation (Art. 16(a)), the provision of legal and technical expertise (Art. 16(b)), the participation in the technical work (Art. 16(c)), and the promotion of European standards and European standardisation deliverables (Art. 16(d)). The funding is provided by the Commission and EFTA; the latter provides 5% of the total EC/EFTA contribution to the financing of the standardisation activities mentioned above. Therefore, in the next paragraphs, we will refer to ‘EC/EFTA’ financing. In 2013, the EC/EFTA provided financing for the ESOs and the Annex III organisations for an amount of almost EUR 30 million. This amount declined significantly in 2014, to some EUR 20 million, despite the new operating grants assigned to the Annex III organisations under the Regulation (and not ready in 2013, except that for SBS). The decline in funding is explained by the 2014 reduction by the Budgetary Authorities of about 25% for standardisation under the negotiations for the Multiannual Financial Framework (MFF) 2014-2020. To align to the new MFF 2014-2020, new Framework Partnership Agreements (FPA) have been signed with CEN/Cenelec and ETSI which, in addition, introduce the concept of Key Performance Indicators (KPI) to attain higher performance, efficiency and effectiveness for the ESS. The same was done for the Annex III organisations receiving Union financing. With regard to standardisation activities carried out in 2013 and 2014, financing granted in 2013-2014 has supported activities that correspond to the policy objectives of the Commission and are linked to the Union’s priorities as set out in the AUWP. This is an obligation resulting from the Regulation. The ICT Rolling Plan is a source of objectives for ICT standardisation deliverables and includes those of other standards setting organisations like fora and consortia (based on the advice of the ICT Multi-Stakeholder Platform (MSP) set up by Commission Decision (2011/C349/04) of 30 November 2011). In practice, only standardisation priorities taken up under the AUWP or the Rolling Plan for ICT standardisation would be eligible receiving Union financing. The AUWP and the Rolling Plan for ICT standardisation are essential tools to identify standardisation work which has priority also in terms of Union’s financing. The Union financing is targeting standardisation work in areas of public policy (e.g. standards and/or preliminary and ancillary work in the area of consumers’ or workers’ health and safety, equal opportunities for disabled people, air quality, environmental protections, climate change, security), that otherwise would not be funded by industry or the NSBs, or not to the magnitude and with the timing required. Moreover, the support to the Annex III organisations plays a significant role (20% of the EU budget of standardisation is dedicated to the Annex III organisations) in strengthening the representation of relevant stakeholder groups (SMEs, consumers, environmental interests and workers) in European standardisation and pursues its objectives in terms of the inclusiveness of the system. The following table details the budget allocations and grants disbursed in respect of 2013-2015. Table 6: Union financing of European standardisation 2013-2015 Years Operating grants (EUR) Action grants (EUR) Total (EUR) 2013 11.348.972 18.367.127 29.716.099 2014 11.281.161 7.939.993 19.221.154 2015 - - 20.885.000

Source: Data refer to the contracts signed by the Commission (DG GROWTH) in 2013, 2014 and 2015.

The total amount for action grants of 2013 includes an action grant from the Commission (DG Energy) equal to €3.779.513. 5.3 Vademecum on European standardisation in support of Union legislation and policies, The Vademecum gives guidance on the preparation and adoption of standardisation requests in order to comply consistently with Articles 8, 10(2) and 12 of Regulation (EU) No.1025/2012 and ensure proper and wide consultation of the standardisation actors. The Vademecum was originally published by the Commission in 2003 as a response to a Council resolution in 1999 . It was revised for the first time in 2009. The second revision, SWD(2015) 205, reflects actions and priorities of the Standardisation Package (COM(2011)311 final and the requirements of Regulation (EU) No 1025/2012). The objective of the Vademecum is to ensure a common understanding of the role of the Commission’s standardisation requests (Implementing Acts) to the ESOs and the role and responsibilities of the various actors in planning, preparing and executing these requests. It is addressed to all actors of the European standardisation system (ESS) and in particular to Commission officials, public authorities in the Member States and EFTA countries, the ESOs, the organisations identified in Annex III to the Regulation, the national standardisation bodies (NSBs) and all stakeholders in European standardisation. To support Union legislation and policies, the revised Vademecum, together with the Commission’s internal project management tools has been aligned in a new process (as defined in the Vademecum) to be used when standardisation requests are needed and relevant. Concerning the standards setting process itself, which is fully under the responsibility of the ESOs, the Vademecum provides guidance on the expected quality of requested deliverables and sets out reporting and transparency principles. 6. Answers to the evaluation questions The present evaluation investigates the overall functioning of the ESS, a key element of which is the progress with the implementation of the Regulation. However, given the recent entry into force of the Regulation, a full ex post evaluation of the Regulation cannot be achieved due to the fact that the full cycle of the standardisation work requested by the Commission since 2013 has not yet been completed and thus the answers provided to the evaluation questions discussed in this section must be considered in that same context and within that constraint. 6.1 Effectiveness 6.1.1 Objectives As part of the new Standardisation Package, launched in 2011, the Communication COM(2011) 311 final defined five strategic objectives, as the guiding principles of the ESS, including the Regulation: speed and timeliness of standards development, support to competitiveness of EU businesses, support to EU legislation and policies, inclusiveness, and support to the competitiveness of EU businesses at global level. The Regulation (EU) No 1025/2012 – that followed the Communication and entered into force in 2013 – established the framework for the effective use of the ESS as support to EU legislation and policies, and introduced several changes, including standards in services, comitology and a stronger focus on stakeholder participation in European and national standardisation activities. The main general objectives of the Regulation are to increase the contribution of standards and European standardisation to a better functioning internal market, to stimulate growth and innovation and to foster the competitiveness of EU enterprises, especially SMEs. In terms of the specific objectives, the Regulation addresses in particular the transparency, timeliness and inclusiveness of the standards development, use of ICT technical specifications developed outside the ESS in public procurement and new rules for financing European standardisation In the following paragraph, an analysis of the progress towards the objectives set out in the Communication, as well as the general and specific objectives of the Regulation, is given. Since the implementation of the Regulation is a part of the overall functioning of the ESS, the objectives of the Communication and the Regulation partly overlap and no strict separation between these objectives is therefore made. 6.1.1.1 Progress towards the strategic objectives and barriers The ESS is achieving most of its stated objectives, contributing to the integration of products and services markets across the Member States and EFTA countries, supporting the elimination of trade barriers, fostering interoperability and setting up an increasingly inclusive, transparent and participatory process. According to the Independent Review, the development time for European standards requested by the Commission has been significantly reduced during recent years. The development time of the Commission requested standards went from 5 years on average in 2009 to 3 years in 2013. The factors which are helping to reduce this time include: The Framework Partnership agreements (FPAs) and Operating Grant agreements (OGs) between the Commission and the ESOs which aim at further reducing the development time of standards; The OGs include key performance indicators and, among these, 50% reduction in the standard development time by 2020 (as mentioned in the COM 311 (2011);

The introduction of the AUWP and the revision of the procedure to plan, consult and adopt standardisation requests are expected to contribute to speeding up the standards setting process. Estimates carried out in Phase 1 of the Independent Review show that the timing for the overall development of standards significantly decreased between 2009 and 2013 (no data available for 2014); the average time for the overall development process in 2013 was about three years, against an average of five years in 2009. This average does not take into account any extremes (very long and very short development time frames). The timeframe indicated for ETSI is shortest. However, different milestones were available for the computation of data, and the figure below should not be used for comparison between the ESOs. Figure 4 - Development time for home grown - 1st edition standards (extremes excluded (5%)) Source: Computations based on CCMC Database – ETSI Database Data limitation: 5% extreme (highest and lowest) were excluded to avoid bias from erroneous datasets and from extreme cases, not necessarily reflecting current performance of the system. Two out of three standards have a development time lower than three years, and the average development time mentioned above is highly influenced by a few standards whose development started a long time ago (see figure over). Figure 5 - Home- grown 1st edition standards made available between 2009 and 2013, by date of decision on Working Item (WI) Source: CCMC Database – ETSI Database Data limitation: 5% extreme (highest and lowest) were excluded to avoid bias from erroneous datasets and from extreme cases, not necessarily reflecting current performance of the system. Looking at the detailed steps of the development process, average EN drafting time is estimated to be 280 days, while enquiry and vote last on average 145 days and 61 days, respectively. The difference between the sum of these values and the average development time is due to a range of administrative activities such as preparation of public consultation, need for multiple consultations, and/or time spent in other activities such as translations. The time needed for these administrative activities seems to be the main obstacle to further improve the performance of the ESS in terms of timely delivery. In the recent past, the ESOs introduced several changes to improve the timely availability of the standards developed upon EC request, also as a consequence of the 2011 Communication. These changes range from general measures aimed at improving the management of the work (a progressive shift from paperwork to electronic tools, organisation of web meetings, alerts system on the progress of the standardisation work), to the revision of formal procedures. Further, under the FPA until 2020 signed with the ESOs, Key Performance Indicators (KPIs) have been agreed. One KPI is the time taken to deliver a standard. A yearly threshold is agreed in the annual OG ensuring a continuous reduction in the delivery time of European standards. Not complying with the threshold may give rise to financial consequences for the next OG. This certainly represents a significant progress towards the achievement of a key objective of the ESS: the speeding-up of the process and the improvement of the timely availability of standards. Moreover, under the Regulation, further significant improvements are expected in terms of timely availability of standards requested by the Commission, through the early involvement of the stakeholders, the creation of consensus around the standardisation requests under preparation, and the improvement in the quality and detail of the requests issued . As to the effective inclusiveness of the ESS, this was one of the central points of the reform package launched in 2011, and progress in this respect has been achieved. Firstly, the Regulation (in its Annex III) has recognised the support to representatives of social and societal stakeholders and SMEs for their participation in standardisation activities .

Secondly, the four stakeholder organisations complying with the Regulation and selected for Union financing (SBS, ANEC, ECOS and ETUC) have undertaken activities in 2013 and 2014, mainly focused on the training of experts, the participation in ESO Technical Bodies, and the setting up of tools for communication and awareness raising (see par. 2.5 of the Article 24 Study). Moreover, Annex III organisations take part in the work of the ESOs at a policy and technical level with different levels of involvement and participation (see par. 2.3 of the Article 24 Study). The implementation of the Regulation suggests that the number of NSBs/NSOs where the four stakeholder groups (SMEs, consumer organisations, social and societal stakeholders in national standardisation bodies) are represented has been quite stable or increasing between 2013 and 2014 with a particularly good track record for ETSI. In particular, SMEs appear to be represented, at both policy and technical level, in most of the NSBs, while the other stakeholders take part to a lesser extent.

In the framework of the implementation of the Regulation, steps have been taken to improve the participation of SMEs and underrepresented stakeholders. Most of the NSBs facilitate the participation of SMEs in the national technical bodies, especially by allowing SMEs to participate without being members of the NSB, and by providing for special rates for the participation in standardisation activities (see par. 2.5 of the Article 24 Study). However, according to key stakeholders, there are still difficulties for SMEs in taking part in standardisation and/or to make use of standards. Similarly, and despite the improvements introduced, the participation of stakeholder organisations, representing consumers’, workers’, SMEs and environmental interests, is still challenged by some limitations at the European and international levels, raised by the Annex III organisations during the Independent Review and in their reports on the implementation of the Regulation. These perceived limitations are the following: difficult access to the relevant information and technical documents (due to the scope of the access rights on the electronic systems of the ESOs and the technical committees), the limited formal status of the Annex III organisations in any of the ESOs affects their visibility and the limited understanding of their role by other actors in the ESS . Moreover, looking at the last strategic objective (“support to competitiveness of EU businesses at the global level”), European standards are increasingly aligned with international standards, thanks to the active use of cooperation agreements between the ESOs and their international counterparts for the development of joint work or the endorsement of international deliverables. In particular, data regarding CEN-ISO cooperation shows that over the past eight years 35% of work items have been developed under the Vienna Agreement , while 77% of Cenelec work items having been developed under the Dresden Agreement, in cooperation with IEC. Although no comparable data is available for ETSI, the mapping between ETSI technical bodies and those of ITU shows a high number of active linkages between ETSI technical bodies and study groups and those of ITU . 6.1.2 Transparency The introduction and implementation of the Regulation since 2013 has contributed to the progress which has been made to the effectiveness of the ESS, most clearly in relation to the inclusiveness and associated transparency of the system. As evidenced in tables 7 and 8, there has been a substantial increase in the representation of certain key, previously under-represented stakeholders within the system. This is particularly so in the case of social and societal stakeholders and SMEs. This increased participation will contribute, in due course, to a more cohesive and legitimate system where the different actors have a better understanding of their respective roles. This is also underlined by the funding of, better status for and participation in ESO work (at policy/technical level) by Annex III stakeholder representative groups which was introduced by the Regulation. Taken together this improved status and participation should ensure that, crucially, information flows more effectively between the Commission, the ESOs, the NSBs and stakeholders who were less well informed previously. 6.1.3 Addressing legal ambiguities The Regulation harmonises certain definitions, the process for adopting standardisation requests, conditions for publishing references of harmonised standards in the OJEU, the process for formal objections and equal criteria for grant funding to SME and stakeholder organisations. These provisions contribute significantly to the achievement of the specific objectives of the Regulation, increasing in particular the transparency and inclusiveness of the standards development, and appear to be well received by the majority of stakeholders. 6.1.4 Use of ICT technical specifications in public procurement The lack of European and international ICT standards had created a permanent problem for public procurement. This could not be directly resolved by other means than the identification procedure for ICT technical specifications established by the Regulation. The option exists to use identified ICT technical specifications in public tenders and this is expected to counter the tendency towards market fragmentation and to have a positive impact on the Single Market. Correspondingly, the referencing of ICT technical specifications enhances the competitiveness of enterprises, in particular the European ICT industry, where officially recognised specifications increase stability and provides an incentive to invest. So far, the Commission has decided that six ICT technical specifications are eligible for referencing in public procurement (Annex 12). Further ICT technical specifications eligible for the use in public procurement are being identified by the European Multi-stakeholder Platform on ICT standardisation set up in accordance with the Commission Decision C(2011) 8600. 6.1.5 Economic impact of standardisation: estimates from national studies Finally, regarding the overarching strategic objective of "Support to competitiveness of EU businesses", this section focuses on what can be established from the estimates made in national studies. Particular use is made of the estimates detailed in the Centre for Economics & Business Research (Cebr, June 2015) report, The Economic Contribution of Standards to the UK Economy, (hereafter, the 'Cebr report') , , and as summarised in a 1 July 2015 Cebr presentation to the Friends of Standardisation Roundtable. The figure below illustrates the complexity of the manner in which standardisation contributes to competitiveness and thereby economic growth - and the different pathways involved. Figure 6 - How do standards contribute to economic growth? Source: Cebr presentation to the Friends of Standardisation Roundtable (2015, p.3)

In light of the complexity of the pathways by which standardisation impacts on competitiveness and growth, it is worth reviewing the methodology employed in the Cebr report. The report (2015, p.7) states that it represents "the most comprehensive study of the economic benefits of standards to UK businesses to date" and that it (ibid) "adds to that important work [a previous 2005 DTI study] by updating the estimates for the impact of consensus standards on productivity at the national level, using a similar methodology and covering the period from 1921 to today". The Cebr report examines the economic impact of standards from two perspectives: 1.an empirical analysis of the macro-economic impact of standards on the UK economy; and 2.a micro-level analysis on how the use of standards and participation in the standards development process produce financial and other benefits for individual companies. In relation to the first perspective, the Cebr report measured the impact of standards empirically at the national (macro-) level by carrying out an econometric analysis to determine the relationship that standards have with productivity and to quantify the value of their contribution to the UK economy. The Cebr report (p.13) also explains that: "The macro-level estimate for the impact of standardisation is based on average productivity growth for the whole economy, incorporating both high and low productivity growth sectors. If only high productivity growth sectors were covered - as was the case in the sector-level analysis - then estimated impacts would likely have been larger." In relation to the second perspective, a survey was carried out of 527 UK companies (all survey respondents used standards though 68% were not involved in their development) and this was combined with in-depth interviews/case studies in seven key sectors. It should be noted (p.12) that: "the micro-level analysis targeted sectors that were identified as the most standard-intensive in the UK economy. These sectors, which represent 25% of the UK non-financial business economy are also sectors that have experienced the strongest productivity growth over the past ten years – productivity in manufacturing grew by 19.7% between Q1 2005 and Q3 2014 compared to just 4.9% growth for all sectors over the same time period." As mentioned in relation to the methodology employed, the Cebr report examines the economic contribution of standards to the UK economy from both a macro- and micro-economic perspective. The following selected findings were identified with a view to highlighting those UK economic impacts caused by standards which would be most relevant on a trans-national basis:

The results of the econometric estimation show that the contribution of standards to productivity growth over the period 1921 – 2013 has been positive and statistically significant; Standards are associated with approximately 0.69 percentage points of the average labour productivity growth of 1.8% a year. This equates to the suggestion that standards supported 37.4% of labour productivity growth and 28.4% of GDP growth between 1921 and 2013; For the purposes of expressing the findings in monetary terms, if it is assumed that the estimated impact is constant over time, standardisation at a national (i.e. UK) level would be associated with approximately £8.2 billion (€10.4 billion) of the £29.0 billion (€36.8 billion) of GDP growth recorded in 2013 (2014 prices); Results from the micro-level analysis provided evidence to support the macro-level findings with 48% of companies surveyed reporting a net benefit from standards. This finding was relatively consistent across industries though larger businesses were more likely than SMEs to report this benefit; The report states (p.10) that "one of the most important economic roles of standards is in promoting and supporting international trade" and reports that standards have been hugely influential in boosting the sales of UK products and services abroad, with reported impacts averaging 3.2% per annum in additional exports. This impact on exports ranged from 0.3% in the energy sector to 9.9% in the food and drink manufacturing sector; The results at sector level show that the impact of standards on annual turnover range within sectors from 1.7% (aerospace and defence) to 5.3% (food and drink manufacturing). The Cebr report (see table over) also provides a very useful overview of the findings of several national studies including their own which finds that "while not directly comparable, particularly due to the different time periods analysed…reveals potentially interesting insights about effect of standards on productivity."

Table 9 – Comparison of summary results from national studies Country France Canada Germany Germany UK UK Organisation and publication year AFNOR (2009) Standards Council of Canada (2007) DIN (2000) DIN (2011) DTI (2005) Cebr (2015) Period of analysis 1950-2007 1981-2004 1961-1990 2002-2006 1948-2002 1921-2013 Estimated function GDP Output Labour Productivity GDP Output GDP Output Labour Productivity Labour Productivity Elasticity of stock of standards 0.12 0.36 0.07 0.18 0.05 0.11 Share of labour productivity growth, % 27.1 17 30.1 - 13 37.4 Growth rate of GDP growth, % p.a. 3.4 2.7 3.3 - 2.5 2.4 Share of GDP growth, % 23.5 9.2 27.4 - 11.0 28.4 Contributions of standards to GDP growth, % points 0.8 0.3 0.9 0.7 0.3 0.7 Source: Cebr report (2015, p.39) The estimated contribution of standardisation to the average growth rate of GDP per annum of the UK (2.4%), Germany (3.3%) and France (3.4%) was in the range of 0.7 percentage points (UK) to 0.9 percentage points (Germany), with France at 0.8 percentage points. With regard to the robustness of the Cebr report's findings with regard to the macro-economic impact of standardisation, several limitations were identified and appropriate caution was urged. However, it may be noted that the estimated contribution of standardisation to the average growth rate of GDP per annum of the UK (2.4%), Germany (3.3%) and France (3.4%) was within a consistent range - of 0.7 percentage points (UK) to 0.9 percentage points (Germany), with France at 0.8 percentage points.

In conclusion, all three Member States (to which reference are made to above) are large developed economies with a tradition of industrialisation. Their combined GDP constitutes approximately one half of total EU GDP and these national studies therefore do deserve attention. Equally, it is acknowledged that comprehensive estimates on the impacts of standardisation at the overall EU level are missing and that the Council Conclusions on Single Market Policy (adopted by the Competitiveness Council on 2 March 2015) invited the Commission to (paragraph 9) "finalise the Independent Review and analyse the impact of standardisation on the economy taking into account the interest of all the parties". Ideally, therefore, a study might be commissioned addressing the competitiveness implications and economic impact of standardisation across the EU, building on the Cebr report as well as the previous econometric studies (DIN 2000, Blind 2004, DTI 2005, AFNOR 2009 and 2016) in several EU countries (France, Germany, United Kingdom). However, this option would have to be considered carefully in light of the significant financial resources involved and the substantial time-allocation (anticipated to be in the region of 24 months) required for such a European-wide study. 6.2 Conclusions regarding effectiveness There are over 4,000 harmonised standards requested by the Commission out of over 20,000 CEN and Cenelec European Standards and 450 harmonised standards out of over 35,000 ETSI technical specifications. In the context that the ESS operates with so many standards and different categories of standards (national, European, international etc.), it is vital that a focus is maintained on the effectiveness of the system. Significantly, the Regulation opened new perspectives for EU financial support to the European Standardisation Organisations (ESOs). The new financial support rules aim at driving continuous improvement in the ESOs' performance by setting deadlines within Commission requests for standards and by making funding conditional on the ESOs' fulfilling related criteria. This is a significant improvement and the limited experience collected so far clearly indicates that these new measures will substantially contribute to the timely delivery of standards requested by the Commission. The Regulation has also increased the transparency of the ESS - in particular, the transparency of the development of the Commission's standardisation requests in support of EU legislation and policies. Crucial elements in this respect are the introduction of comitology, the launch of a publicly accessible notification system and the introduction of new reporting requirements for the relevant standardisation actors. Steps have been taken to improve the participation of SMEs and underrepresented stakeholders. Nevertheless, there is a room for introduction of additional measures to improve the inclusiveness. The Regulation also facilitates the exploitation of the potential of innovative ICT technologies and contributes to the realisation of the Single Market. Finally, a wide-ranging study would be needed to assess the overall effectiveness of the ESS with regard to its impact on European economic growth. 6.3 Efficiency 6.3.1 Efficiency of the European standardisation system (ESS) and Regulation This section mainly encompasses the ESS but also, to a certain extent, the implementation of the Regulation to date. The Independent Review of the ESS addressed the efficiency of the system from an overall perspective, and provided an assessment of the main elements of efficiency/inefficiency. The results showed that the ESS is generally perceived as a complex process by the stakeholders (based on the replies to an online survey and the information collected through interviews). Its efficiency is partly limited by bottlenecks in the process specific to standardisation requests from the Commission, the differences of working methods between actors involved (Commission, ESOs and NSBs), and difficulty accessing information on standardisation activities by all the stakeholders involved. Issues have been raised, in relation to the implementation of the Regulation by ESOs and/or Annex III organisations, and the execution of standardisation activities, in particular, in terms of: Timing and predictability and for the signature of the grant agreements: For their first generation, significant delays between the proposals from the ESOs and the Annex III organisations and the finalisation of operating grants are often recorded. These delays result in the need to perform work before financing is received, cause issues in finalising contractual engagements or slow down the commencement of the work; Complexity of procedures and control processes linked with Union financing (applications for financial support, the technical and financial reporting for operating and action grants and the provision of related supporting documents); Reporting associated with the overall activities undertaken under the Regulation, on the implementation of the Regulation (according to its Art. 24), is perceived by the ESOs as an additional constraint; Efforts required by the ESOs and the Annex III organisations in applying for the operating grants and for reporting on their execution. These issues, in particular the reporting requirements according to Art. 24 of the Regulation and procedures linked to operating grants are perceived as administrative burden by the ESOs. However, this perception is not supported by available data. Estimates carried out by the Article 24 study indeed show that a certain effort is required by the ESOs and the Annex III organisations in applying for the operating grants (i.e. the drafting of the annual plan), and for reporting on their execution. The estimates on administrative costs related to operating grants suggest that these represent a small share of the total amount received (around 1% for the ESOs, and between 2% and 5% of the amounts received by the four Annex III organisations in the form of operating grants). In 2013 and 2014, several actions have been undertaken by the Commission to improve the efficiency of the system and to simplify standardisation activities. The new Vademecum on European standardisation in support of Union legislation and policies has been defined as a tool, addressed to all the parties participating in the system, and aimed at clarifying the mechanisms for the Commission’s standardisation requests.

The Commission has also approved in 2015 a decision on 'lump sums': in relation to the development or revision of European standards, and the verification of quality and conformity to EU legislation and policies. The grants will take the form of reimbursement of the eligible costs declared by the ESOs on the basis of lump sums. Based on these decisions, the reimbursement of costs will be based on unit costs, established in the decisions, and the number of days necessary to implement the action (estimated in the ESOs proposals, and assessed and approved on case by case basis by the Commission evaluation committee ). The decisions on 'lump sums' simplify the financing in relation to activities carried out by the ESOs and of direct relevance for the European standardisation policy. This 'lump sums' approach therefore reduces the administrative burden for the standardisation actors. On the basis of those actions the situation is expected to improve – however this requires new internal management of such grants by the ESOs and result-oriented thinking to utilise such grants effectively. 6.4 Conclusions regarding efficiency Taking into consideration the acknowledged complexity of the ESS and the relatively recent implementation of the Regulation as well as the longer timeframe required before standards development processes and related activities can be assessed, the current data does not allow for a fully reliable analysis of the efficiency of the ESS as influenced by the activities supported by the Commission and within the new framework of the Regulation. While there is a perception by the ESOs and the Annex III organisations that the Regulation has increased the administrative burden on their part, this perception is not supported by available data. Still, perception is a sufficient reason for the Commission to act beyond the steps already undertaken (Vademecum; lumpsums). Under the next evaluation planned by Article 24 in 2020, it is anticipated that sufficient data will be available to make a fuller assessment. However, the foregoing comments should be placed in the overall context that (as previously discussed in relation to the Cebr report) there are complementary factors influencing the economic impact of the ESS which cannot be captured without a more wide-ranging study. 6.5 Relevance 6.5.1 Relevance of the intervention The Impact Assessment (IA) carried out in 2011 identified the following needs from stakeholders, as further explained in the previous section 2 "background of the initiative": - Speed: need to reduce the time taken by the standardisation process for standards developed at the request of the Commission - Inclusiveness: need to ensure that SMEs and societal stakeholders are adequately represented in the standardisation process, especially for standards developed at the request of the Commission - Need to broaden the use of ICT standards and thus enhance interoperability through a more integrated European public procurement market for ICT products and services, especially in connection with the establishment of an 'e-Internal Market' - Need to remove ambiguities in the existing legal framework. Those needs are still valid among stakeholders and the Regulation, although relatively recently implemented, seems to address them. In addition, the Regulation also significantly improves the transparency of the ESS. Additionally, the ESS is faced with a fast-changing environment and the need to adapt to the economic and social dynamics. The Independent Review, based on the findings related to each strategic objective and on the conclusions of the 2013 JRC study “How will standards facilitate new production systems in the context of EU innovation and competitiveness in 2025?”, focused on two main trends which are affecting the ESS: (i) the globalisation of the economy and (ii) the pervasive role of ICT systems. In order to ensure the suitability of the ESS for the future, the following points have been identified: Increasing the importance of standardisation at the global level and the need for better coordination between standardisation bodies at all levels (national, European and international); Increased role of service standards: standardisation in services appears to be at a very early stage, while the increasing relevance of this economic sector calls for a better understanding of the needs and potentialities for standards development. Moreover, services and products are becoming more and more interconnected, with the advancement of ICT systems and the rise of products bundled with services (e.g. the rise of smart objects and smart environments which combine products with accompanying services); Increased relevance of IPR/patents in standardisation processes: as ICT concentrates a significant part of patent activity, the digital enablement of the economy might lead to increased patent declarations related to standardisation. As a consequence, solid patents declaration procedures and policies need to be defined in order to ensure transparency about existing IPR, for patent holders and for implementers. Enforcing the protection of IPR while ensuring transparency is an important element to enable innovation among European businesses and to promote the development and use of standards based on fair conditions . Within the overall strategic objectives established by the ESS, these elements represent points for future policy development. 6.5.2 Relevance of the standardisation activities receiving Union financing

Taking into account the objectives established with the standardisation package and the overall Commission priorities in terms of standardisation policy, Article 24 of the Regulation also seeks to assess the relevance of “standardisation activities receiving Union financing in the light of the requirements of Union legislation and policies”. The analysis has been aimed at assessing to what extent the EC/EFTA financial support allocated to the ESOs and the Annex III organisations is relevant to EC priorities in the field of standardisation, if there is a rationale for financing of these specific activities, and how these activities contribute to the Commission’s objectives in the standardisation policy. The following three elements have been investigated in more detail below: (1) typology of activities supported and links between the activities funded in 2013 and 2014 and the priorities defined by the Commission in the Annual Union Work Programme (AUWP) and/or the ICT Rolling Plan; (2) contribution of the activities funded to the implementation of the Regulation and the achievement of the Commission’s objectives; (3) presence of a clear rationale for Union financing of the activity, i.e. whether the Commission is funding actions that would otherwise not happen, or not to the same extent/with the same timing . (1)Typology of activities supported in 2013 and 2014 and consistency with the AUWP In 2013, the highest share (62%) of Union funding has been devoted to action grants (intended as encompassing “grants for an action” and translation grants), while the financial allocations for 2014 are hardly representative, since no fund for the development or revision of European standards (under Art. 15 (a)) has been awarded pending the approval of the decision on the methodology for the calculation of the lump sums. In summary, the detailed activities for European standards revision or development and the preliminary and ancillary work supported by the EC/EFTA between 2013 and 2014 appear to be, in all cases, of relevance to the Commission policy priorities, as spelled out in the annual Union work programmes, and directly related to Union legislation. For the Annex III organisations, the operating grants support all the activities related to the participation of organisations representing stakeholders at the European level. In this view, also activities supported with the operating grants appear directly relevant to the strengthening of the ESS, by contributing to the improvement of the overall system and of its inclusiveness through the participation of relevant stakeholder categories. (2) Rationale for the Union financing Most of the standardisation work co-financed by EC/EFTA with action grants is identified in the AUWP or in the Rolling Plan for ICT standardisation and, in many cases, is related to Commission’s standardisation requests. This link suggests how the Union financing is mainly targeting areas of public policy (e.g. consumers’ and workers’ health and safety, equal opportunities for disabled people, air quality, climate change), supported by the Commission and related to fields of work that otherwise would not be completely funded by industry, or not to the magnitude and with the timing required. Regarding the Annex III organisations, their representatives agree that the activities currently funded with the operating grants would not happen in the absence of Union financing. On the one hand, the Annex III organisations have limited financial resources, coming from the annual fees of the members or entirely from public funding (as is the case for ANEC) that would not allow them to fully carry out their work in relation to European standardisation. On the other hand, there is a lack of awareness or difficult access to standardisation by SMEs, consumers, workers and environmental stakeholders. Standards are voluntary and the knowledge on their economic and social impacts appears limited among these stakeholders categories: these two factors make it difficult to draw the attention of and to promote available funding among stakeholders. 6.6 Conclusions regarding relevance There are three strands to the ongoing relevance of the Commission's standardisation activities: the four policy objectives identified in the IA and which are supported by the Regulation continue to be relevant and will continue to be monitored; in addition, the ESS is faced with a fast-changing environment and the need to adapt to economic and social dynamics. In particular, the two main trends which are affecting the ESS: the globalisation of the economy and the pervasive role of ICT systems; Union's funding of standardisation activities continues to be relevant to support EU policies.

At this stage of the implementation of the Regulation, and in the context of the studies and/or reports that have been finalised recently by the Commission (DG GROWTH and the JRC), the relevance of the standardisation activities and continued policy development is justified. 6.7 Coherence Standardisation is interlinked with a range of other Union policies, as it plays a key role in the creation of the Single Market, the interoperability of products, services and processes, the efficiency of production, a high level of consumer and environmental protection, innovation and social inclusion. The framework established by the Regulation and the standardisation activities promoted contribute to wider Union’s policy objectives, primarily in the area of the Single Market and Digital Single Market. In particular, the recent Commission Communication for a Digital Single Market (DSM) identifies the essential role of standardisation, with focus on ICT standards and interoperability. The current system is deemed relevant to support the priorities fixed by the EC in its new strategy for the DSM. In a broader perspective, standards can also help achieve at least five of the ten Commission priorities: boosting jobs, growth and investment; the connected digital single market; a resilient Energy Union; a deeper and fairer internal market; and the EU being a stronger global actor. The Commission only considers financing specific standardisation deliverables when they are part of the annual UAWP for European standardisation. The AUWP is drafted in line with the President Juncker priorities (see tables 10 and 11 in Annex 13 and 14). Through the establishment of the AUWP consistency is ensured and overlaps with other policy domains, in the context of standardisation, is excluded through close monitoring of the process inside the Commission as well as externally through consultation with stakeholders (the Committee on Standards, the ESOs, the Annex III organisations and all other relevant stakeholders). Coherence is further strengthened through transparency as this programme is published at every stage from the draft through to the final version on the notification system as required by Article 12 of the Regulation. 6.8 Conclusions regarding coherence Inasmuch as standardisation is acknowledged to play a fundamental and longstanding role in the development of the Single Market, it is inherently coherent with a wide range of other related EU policies. The framework, established by the Regulation, and the standardisation activities it supports and seeks to develop contribute to broader Union policy objectives, primarily the Single Market (and the Digital Single Market). On a practical level, coherence is achieved through the use of the AUWP (which is line with the President Juncker priorities), and subsequently through close monitoring of the process inside the Commission as well as through interaction with the Member States and relevant stakeholders. 6.9 EU added value

6.9.1 The added value of the ESS for stakeholders European standardisation plays a key role in the creation and functioning of the Single Market for products and services by gradually harmonising conflicting national standards that may create technical obstacles to entering national markets. They are thus eliminating possible barriers to trade. Such voluntary harmonisation may contribute to increasing the competitiveness of European industry but only when European standardisation is wide-ranging, internationally recognised and easy to comply with (especially by SMEs). Regulating stakeholder representation, transparency of work programmes of standardisation organisations and transparency of draft standards by the Regulation ensures framework conditions for a fair standardisation system. In most cases, the Commission's requests for European standardisation supports Union’s legislation (based on Article 114 of TFEU, establishing the Single Market for products and for services). The Regulation also allows the Commission to request standards in support of the Union’s other policies thus enabling the promotion, case by case, of voluntary standardisation beyond Article 114 of TFEU. Considering the nature of the policy area measures at European level are most appropriate. Correspondingly, measures to improve standardisation at national level cannot deliver EU added value as such. However, it is necessary to rely on synergies between the EU and the national level to deliver that. The development of harmonised standards, the removal of trade barriers and the creation of a Single Market represent by far the major added value of European standardisation. As such, the ESS assumes a direct EU relevance and pursues objectives that could be hardly achieved at national level with the same level of effectiveness. Companies, in particular, benefit from a reduction in transaction costs, an increased level of interoperability and the development of a common technical language within the Single Market. Lack of European and international ICT standards had created a permanent problem for public procurement and could not be directly resolved by means other than the identification procedure established by the Regulation. The main advantage of this procedure is that the possibility of referencing selected fora and consortia standards for procurement purposes on a firm legal basis is expected to counter the tendency towards market fragmentation and to have a positive impact on the Single Market (especially for businesses delivering goods or services complying with these fora and consortia standards). Correspondingly, the referencing of fora and consortia standards enhances the competitiveness of enterprises, with the creation of a competitive advantage for the businesses concerned. This relates, in particular, to the European ICT industry. Also, the fact that these specifications are officially recognised reassures enterprises about their stability and provides an incentive to invest. 6.9.2 The added value of the standardisation activities funded by the EU Ongoing financial support at EU level, rather than at national level for standardisation activities (for the ESOs and to enhance stakeholder participation), is crucial in ensuring that the ESOs also elaborate the standards needed to support EU legislation and policies. Looking at the standardisation activities supported in the framework of the Regulation, the financial allocations (action grants) to the ESOs only contribute to the development and translation of European standards of direct relevance for EU legislation and policies. Reduced support to the general activities of the ESOs (operating grants) would likely result in a reduced responsiveness of the ESOs to the Commission requests and, in a wider perspective, in a much weaker link between European standardisation and the relevant EU priorities. The financial support available annually is used effectively to create better EU added value as it support initiatives in support of the development of the ESS that would not otherwise be undertaken. However, consideration could be given in future as to how the available budget could be more performance-based (focusing notably on the ratio between operating grants and the action grants linked directly to work requested by the Commission). 6.10 Conclusions regarding EU added value The ESS provides a direct EU added value via the development of the Single Market and pursues objectives that could not be achieved at national level. Through the level of harmonisation it brings, the Regulation provides added value by breaking down technical, non-tarriff barriers to trade within the EU (and EFTA). The benefits generated are particularly important for SMEs and societal stakeholders. This represents the major added value of European standardisation. 7. Overall conclusions Regulation (EU) No 1025/2012, providing the legal basis of the ESS, has applied since 2013. Overall, no major issue has been raised concerning the application of the relevant provisions of the Regulation by the ESOs, their national members (National Standardisation Bodies or NSBs and National Standardisation Organisations or NSOs) or stakeholders organisations eligible for Union financing (the Annex III organisations). In this framework, the standardisation activities supported by Union financing in 2013-2014 (and in the period immediately before) are explicitly and directly linked to the policy objectives of the Commission, and linked with EU priorities as set out in the Annual Union Work Programme and/or the ICT Rolling Plan. In a broader perspective, the current standardisation package (which took effect in January 2013) introduced a set of strategic objectives to enhance the contribution of standards to the growth of the European economy and the creation of an increasingly inclusive and participatory standards development process. The achievement of these stated objectives contributes to the harmonisation of products and integration of the services market (where appropriate) across the EU Member States and EFTA countries, supports the elimination of trade barriers, fosters interoperability and sets up an increasingly inclusive system. A more far-reaching exercise than this essentially procedural evaluation would be required to fully support conclusions on the progress achieved by the ESS. Such an evaluation would have to measure the impact of European standards on the EU economy rather than the objectives, efficiency and effectiveness of the process of creating European standards. Moreover, the four stakeholder organisations, selected according to criteria listed in the Regulation and eligible for Union financing (SBS, ANEC, ECOS and ETUC), have mainly focused on the training of experts, participating in ESOs' Technical Bodies, and the setting up of tools for effective communication and awareness raising. Moreover, these organisations took part in the work of the ESOs at a policy and technical level, with different levels of involvement and participation. Steps have been taken to improve the effective participation of SMEs and underrepresented stakeholders. Most of the NSBs also facilitate the participation of SMEs and the national technical bodies allow SMEs to participate without being members of the NSB, and provide special rates for participation in standardisation activities. For efficiency assurance, the Vademecum on European standardisation in support of Union legislation and policies has been defined as a tool addressed to all the parties participating in the system and aimed at clarifying the mechanisms for and role of the Commission’s standardisation requests. Like previously mentioned, the Commission has also, in 2015, approved a decision on the use of 'lump sums'. These will be used in relation to the development or revision of European standards, and the verification of quality and conformity of EU legislation and policies. The grants will take the form of reimbursement of the eligible costs declared by the ESOs. The decisions on 'lump sums' is intended to make financing more efficient in relation to activities carried out by the ESOs. In order to ensure the relevance of the ESS for the future the following topics of attention have been identified:

Increasing need for standardisation at the global level; Increased role of service standards; Increased relevance of IPRs and patents in standardisation. Taking into account the objectives established for the standardisation package and the overall Commission priorities as to standardisation policy, the evaluation of standardisation activities receiving Union financing in the light of the requirements of Union legislation and policies has been aimed at assessing to what extent the EC/EFTA financial support (allocated to the ESOs and the Annex III organisations) is relevant to EC priorities. In the field of standardisation, these activities are relevant above all to the development of the Single Market and would not be pursued at national level. Therefore, the need for this ongoing support is confirmed. Standardisation is interlinked with Union policies: it is designed to play a coherent role in the development of the Single Market. The framework established by the Regulation and the standardisation activities is designed to contribute to wider Union policy objectives, primarily the Single Market and the Digital Single Market. Through the establishment of the AUWP, including all EU policy priorities, coherence is targeted through close monitoring of the process inside the Commission as well as through interaction with stakeholders (Committee on Standards, ESOs, Annex III organisations etc.). The development of harmonised standards target the removal of trade barriers in the Single Market, respresenting a major added value of European standardisation. As such, the ESS assumes a direct EU relevance and pursues objectives that could not be achieved at national level. Companies, in particular, can benefit from a reduction of transaction costs, an increased level of interoperability and the development of a common technical language within the European Single Market. The ESS involves a large number of actors who have different interests, and bottlenecks reduce its effectiveness and efficiency. This evaluation has identified several areas for further improving the achievement of the strategic objectives of the process, strengthening the overall governance of the system and its link to identified challenges. It has also identified needs for simplification in relation to reporting requirements and the procedures related to Union financing described in the Regulation. Those issues are identified in Annex 1 to the Article 24 Report to the European Parliament and the Council (which report this Commission Staff Working Document accompanies). The Article 25 report which evaluated the impact of the procedure established by Article 10 of the Regulation on the timeframe for issuing standardisation requests concluded that "the new rules under Article 10 of the Standardisation Regulation increase the need for efficient cooperation between the involved actors” [the Commission, the Member States, standardisation organisations, Annex III organisations and relevant other stakeholders], "the short timeframe to gain experience with the new processes does not allow to draw a final conclusion about whether the new rules will result in a permanent and unacceptable increase in the length of procedures for [planning, preparing and adopting] standardisation requests. Thus, there is not enough data to substantiate the need for a legislative proposal to amend Article 10 of the Standardisation Regulation at this stage. The Commission will revisit the situation five years after completion of the present evaluation, in the context of the report that it will present to the Council and to the European Parliament on the implementation of the Regulation, under Article 24(3). In addition to the Article 24 Report, the Council Conclusions on Single Market Policy (in March 2015) invited the Commission to "analyse the impact of standardisation on the economy taking into account the interest of all the parties". As previously discussed, a substantial study on the EU-wide macro-economic implications of the ESS would provide a far more substantive assessment of the efficiency, effectiveness, relevance, coherence and EU added value of the contribution of the ESS. It would also be very relevant in assessing the further development of the system itself and how the Commission should further support EU priorities, especially (though not exclusively) in relation to the Single Market and Digital Single Market.



Annexes Annex 1: List of Acronyms AUWP Annual Union work programme CCMC CEN-CENELEC Management Centre CEN European Committee for Standardisation

CENELEC European Committee for Electrotechnical Standardisation COS Committee on Standards EC European Commission EFTA European Free Trade Association EN European standard ESO European Standardisation Organisation ESS European Standardisation System ETSI European Telecommunications Standards Institute EU European Union FPA Framework partnership agreement hEN Harmonised European standard ICT Information and communication technologies IEC International Electrotechnical Commission ISO International Organisation for Standardisation ITU International Telecommunication Union KPI Key performance indicator MS Member States MSP Multi-stakeholder platform NGO Non-governmental organisation NSB National standardisation body

OJEU Official Journal of the European Union REGULATION Regulation (EU) No. 1025/2012, on European standardisation SME Small and medium-sized enterprises TB Technical body TC Technical committee WTO World Trade Organisation



Annex 2: The actors in the ESS European standardisation organisations (ESOs) CEN, Cenelec and ETSI are recognised by Article 2(8) of the Regulation as European standardisation organisations. • CEN (the European Committee for Standardisation) exercises its standardisation activities in all fields, except for electro-technology and telecommunication; • Cenelec (the European Committee for Electro-technical Standardisation) exercises its standardisation activities in the electro-technical field; • ETSI (the European Telecommunications Standards Institute) exercises its standardisation activities in the ICT and telecommunication fields. Each of the ESOs is an independent, non-profit organisation that operates as a platform composed of two main parts: a “secretariat” and a “network”. The secretariat is the physical organisation and a legal entity responsible, under the statutes of the relevant ESO, for handling all administrative and practical matters of the network and for ensuring that the standards are developed according to agreed-upon (open, transparent, inclusive) processes. The CEN/Cenelec Management Centre (CCMC) in Brussels is the joint secretariat of CEN and Cenelec, while ETSI has its own secretariat in Sophia Antipolis in the south of France. In the cases of CEN and Cenelec, the members of these organisations (legal entities) are the NSBs covering 28 EU Member States, the Former Yugoslav Republic of Macedonia and Turkey and the EFTA countries Iceland, Norway and Switzerland. In the case of ETSI, industry and other stakeholders on a worldwide basis (around 800 members from 64 countries) are also members of the organisation. National standardisation bodies (NSBs)

National standardisation bodies (NSBs) are the organisations (legal entities) that develop and adopt national standards and are recognised at a national level and notified by the Member States to the Commission . These bodies represent the interest of national interested parties in standardisation. Usually, these bodies are also members of CEN, Cenelec and/or ETSI, and ISO and/or IEC. Standardisation matters for a given country are either handled by a single NSB, or more than one NSB (with their respective fields of expertise, similar to ESO fields). NSBs may be either public or private sector organisations, or combinations of the two, and may undertake a range of related activities in addition to standards development and adoption. The NSBs play a pivotal role in the ESS, enabling the interested parties (industry and other stakeholders) to participate in the elaboration and approval of standards at national, European and international (ISO/IEC) levels. International standardisation bodies At the international level, the Regulation in Article 2(9) identifies the following three international standards bodies, which to some extent are the analogues of the ESOs, and where NSBs are also members: • ISO (the International Organisation for Standardisation) performs standardisation activities in all fields, except for electro-technical and telecommunications. ISO currently has members representing national standards bodies of 162 different countries, 117 of which participate as full members; • IEC (the International Electro-technical Commission) performs standardisation activities in the electro-technical field. IEC currently has 59 full members and 23 associate members, representing 82 national committees (NCs); • ITU (the International Telecommunication Union) performs standardisation activities in the ICT and telecommunication fields. It currently has 193 national members and more than 700 “sector members” from industry. Other standardisation bodies A range of other organisations exist, including fora and consortia, which are not national, European or international within the meaning of the Regulation, especially in the ICT field but not restricted to ICT, and develop standards or other technical specifications which may be utilised in the ESS. The ESO may co-operate or have liaisons with such organisations and in case of ICT domain, the output from those organisations may be subject to identification within the meaning of Article 13 of the Regulation. Industrial stakeholders Standards are in most cases initiated, developed and applied by industry and businesses. As such, industry plays a key role in the ESS, and is arguably the most important “actor” in the development and use of European standards. Industry can participate in the development and approval process of European standards through respective NSBs or in the case of ETSI, through direct membership and participation at the European level. The ESS is based on the principle of national delegation (CEN/Cenelec) and on the principle of direct membership (ETSI), wherein: • In CEN/Cenelec, the development of standards is managed by Technical Committee (TCs) comprised of “national delegations”, each of which represents one of the national member (NSB). Where there is sufficient interest, the NSBs may create “mirror committees” or equivalent bodies ('mirroring' the European TCs) at the national level, which enable interested parties (industrial and other stakeholders) to participate in the national discussions during the creation of the standard in order to decide on national positions. This is usually possible once during public enquiry stage. All draft standards remain to be in English, German and French. • In ETSI, the development of standards is based on direct participation of ETSI members, i.e. industrial and other members without representative intermediaries. However, in the case of ENs, final adoption must also be given by the relevant representative NSBs. The work done by ETSI i