EUROPEAN COMMISSION Brussels, 11.1.2018 SWD(2018) 6 final COMMISSION STAFF WORKING DOCUMENT



IMPACT ASSESSMENT

Accompanying the document Proposal for a Council Regulation on establishing the European High Performance Computing Joint Undertaking {COM(2018) 8 final}

{SWD(2018) 5 final}



Table of Contents Preamble 1.Introduction 1.1.Background 1.2.Context 2.Problem Definition 2.1.A brief introduction to HPC and its strategic value for science

and the digital economy 2.2.What is the problem that requires action and its size 2.3.What are the problem drivers? 2.4.Who is affected and in what ways? 2.5.How would the problem evolve? 3.Why should the EU act? 3.1.Why do we need to act now? 3.2.The right to act at EU level 3.3.Subsidiarity 3.4.EU added value 4.Objectives: What should be achieved? 4.1.Overall objectives of the EuroHPC entity and instrument

4.2.Functionalities of the EuroHPC instrument 4.3.Specific Objectives 5.Available Policy Options 5.1.Option 0: Baseline from which the policy options are assessed 5.2.Options considered for implementing EuroHPC but discarded at an early stage 5.3.Option 1: ERIC 5.4.Option 2: Joint Undertaking 6.What are the Impacts of the Policy Options? 6.1.Option 0: Baseline scenario 6.2.Option 1: ERIC 6.3.Option 2: Joint Undertaking (JU) 7.How do the Options Compare? 8.Selection of Preferred Option and how will the EuroHPC JU work 9.How would actual Impacts be monitored and evaluated?



LIST OF ANNEXES -Annex 1: Procedural information -Annex 2: Analysis of the responses received to the targeted Consultation and on the Inception Impact Assessment on the HPC Initiative in Europe -Annex 3: Who is affected by the initiative and how?

-Annex 4: Staff and budgetary estimates for the Euro HPC JU option -Annex 5: HPC and its strategic value for the digital economy -Annex 6: Brief review of the funding sources and budgets of HPC initiatives Worldwide and in Europe



GLOSSARY The below table explains the key terms or acronyms used in this document. ASCR Advanced Scientific Computing Research CEF Connecting Europe Facility CoE Center of Excellence cPPP Contractual Public-Private Partnership DoE (US) Department of Energy DSM Digital Single Market EC European Commission ECI European Cloud Initiative EEIG European Economic Interest Grouping EIB European Investment Bank ERIC European Research Infrastructure ETP European Technology Platform ETP4HPC European Technology Platform for High-Performance Computing Exascale Computing systems capable of 1018 Floating Point Operations per Second FET Future and Emerging Technologies Flop Floating Point Operations per Second FP7 7th EU Framework Programme for Research & Innovation FPA Framework Partnership Agreement GDP Gross Domestic Product H2020 Horizon 2020 Framework Programme for Research & Innovation HPC High-Performance Computing ICT Information and Communication Technology IPCC Intergovernmental Panel on Climate Change IPRs ISV Intellectual Property Rights Independent Software Vendors ITER International Thermonuclear Experimental Reactor JTI Joint Technology Initiative JU Joint Undertaking (as defined by Article 187 TFEU) LEIT Leadership in Enabling and Industrial Technologies MFF Multi-annual Financial Framework NSA (US) National Security Agency NSCI National Strategic Computing Initiative OJ Official Journal of the EU PCP Pre-Commercial Procurement PPI Public Procurement of Innovative solutions PPP Public-Private Partnership PRACE Pre-exascale Partnership for Advanced Computing in Europe Computing power near the exascale performance (i.e. 0.1-0.6 exascale) R&D Research and Development R&I Research and Innovation ROI Returns on Investment SME Small- and Medium-sized Enterprise SRA Strategic Research Agenda SSC (PRACE) Scientific Steering Committee TFEU Treaty on the Functioning of the European Union WP Work Programme



Preamble In this document the terms High Performance Computing (HPC) systems, HPC machines and supercomputers are used indistinctly. The terms leading-class/world-class computers or systems refer to supercomputers that are ranked amongst the most powerful in the world. An important point is the level of computing performance of supercomputers: the next HPC computing frontier is the exascale performance, i.e. supercomputers capable of executing 1018 or 1 billion billion Floating Point Operations per Second. These systems are expected to be built around 2022. The term pre-exascale is used to refer to performance levels close to exascale (i.e. 0.1 to 0.7 exascale). Pre-exascale systems are expected to be available on the market around 2019-2020. 1.Introduction This document explains that High Performance Computing (HPC) is a strategic priority for Europe. The European Commission (EC) and several Member States (MS) have already taken the decision to co-invest in a joint structure that would allow Europe to reach the strategic goals defined in this domain through optimal use of available public funding. The document explores which form of legal and financial instrument would serve best the objectives of this joint HPC structure (called in the sequel the EuroHPC entity) that would start operating as of 2019 onwards and assesses its impact to implement the EuroHPC strategy in Europe. 1.1.Background Europe's scientific capabilities, industrial competitiveness and sovereignty critically depend on access to world-leading HPC computing and data infrastructures to keep pace with the growing demands and complexity of problems to be solved. In 2012, the Communication "High performance Computing: Europe's place in a global race" highlighted the strategic nature of HPC as a crucial asset for the EU's innovation capacity and called on Member States, industry and the scientific communities, in cooperation with the EC, to step up joint efforts to ensure European leadership in the supply and use of HPC systems and services by 2020. On 19 April 2016, the EC adopted the European Cloud Initiative (ECI), as part of its Digitising European Industry strategy. The Communication invited the EC and the MS to work together in the creation of a leading European HPC and Big Data ecosystem, underpinned by a world-class HPC, data and network infrastructure. Such infrastructure would support the EU to become one of the world's top supercomputing powers by realising exascale supercomputers around 2022, based on European technology, ranking among the first three places in the world.





The European HPC Strategy aims at establishing a world-class HPC ecosystem in Europe, acquiring leadership-class supercomputers which secure Europe's own independent HPC technology and system supply, and deploying HPC services for science, industry and SMEs. To be able to implement this strategy we need to coordinate and pool national and European efforts in developing and procuring world-class supercomputers. In May 2016, the Competitiveness Council expressed its political support to HPC followed by the European Parliament in January 2017. The Competitiveness Council took also note of the intention of France, Italy, Luxembourg and Spain to launch a joint project for developing a commercial offer in HPC and big data serving industrial applications. Moreover, the European Council of 28 June 2016 called for swift and determined progress to create the right conditions for stimulating new business opportunities by coordinating EU efforts on high-performance computing; and looked forward to the launch of an important project of common European interest in this field. On 23 March 2017 at the Digital Day in Rome, which was organised as part of the 60th Anniversary celebrations of the Treaty of Rome, seven MS – France, Germany, Italy, Luxembourg, the Netherlands, Portugal and Spain – signed the EuroHPC declaration. They were more recently joined by Belgium, Slovenia, Bulgaria, Greece, Croatia and Switzerland. These countries agreed to work together and with the EC for acquiring and deploying by 2022/2023 a pan-European integrated exascale supercomputing infrastructure called EuroHPC. Other MS and Associated Countries (AC) are invited to sign the EuroHPC declaration. Several countries have already signalled their intention to do so by the end of 2017. The EuroHPC Declaration is an agreement in which the signatory MS commit to work together and with the EC for acquiring and deploying an integrated world-class HPC infrastructure, which will be made available across the EU for scientific communities as well as public and private partners, no matter where supercomputers are located, upraising Europe's scientific capabilities and industrial competitiveness and for jointly procuring and deploying exascale supercomputers accessible from everywhere in Europe based on competitive European technologies. On 10 May 2017, in the Communication on the Mid-Term Review of the Digital Single Market (DSM) Strategy, the EC confirmed its plans to invest on HPC and announced its intention to propose by end-2017 a legal instrument that provides a procurement framework for an integrated exascale supercomputing and data infrastructure. 1.2.Context The EuroHPC declaration and the communication on the DSM mid-term review have soundly supported the decision on establishing the joint EuroHPC entity that allows the co-investment of the Union with MS and ACs to establish an integrated world-class supercomputing and data infrastructure. The EuroHPC entity would permit to coordinate and pool national and European efforts in a framework to procure jointly between the Union and the MS a world class HPC and data infrastructure and their interconnection. In order to be able to procure such infrastructure based on competitive European technology, the EuroHPC entity will also be used to develop further the European HPC ecosystem, i.e., a reinforced technology supply chain, a richer applications offer in various sectors and the tools necessary to provide these applications as an HPC Cloud service across Europe. This includes support to Research and Innovation (R&I) on both hardware and software required for building competitive exascale machines as well as support to the development of skills needed for reaping the benefits of investing in such leading infrastructure. There is an urgent need to act now, driven by the triple need to: (i) procure and deploy in Europe in competitive timeframes a world-class pre-exascale HPC infrastructure; (ii) make it available to public and private users for developing leading scientific and industrial applications that would foster the development of a broad pre-exascale ecosystem in Europe; and (iii) support the timely development of the next generation European HPC technologies and their integration into exascale systems in order to be ready to procure them in competitive timeframes with respect to our world competitors. These three objectives are within reach if the EuroHPC entity could be set up and start operating in 2019. The entity would not need extra funds from the current Multiannual Financial Framework (MFF), as it will draw funds from the budgets already committed for HPC activities in the different work-programmes (WPs) of the last two years of Horizon 2020 (H2020) and the Connecting Europe Facility (CEF). A total budget of approximately EUR 1 billion would be available: an EU contribution of around EUR 486 million matched by a similar amount from the MS/AC. This would ensure the operations and payments of all the activities that the EuroHPC entity would launch at the latest by the end of 2020 until their termination around 2025/2026. The EuroHPC entity may also receive financial support from the next MFF. Should this be the case, the entity would be ready to fulfil further the objectives of the European HPC strategy. This would imply in particular the possibility for the entity to procure exascale HPC systems based on European technology; support the development of a thriving exascale ecosystem in Europe; and prepare for the next generation of HPC technologies and their link to quantum computing.



2.Problem Definition 2.1.A brief introduction to HPC and its strategic value for science and the digital economy HPC is a branch of computing that deals with simulation and modelling of scientific and engineering problems and with data analytics that are computationally so demanding that computations cannot be performed using general-purpose computers. Today, these computations typically run on very powerful systems with highly parallelized computing units of hundreds of thousands or millions of processors. Those computers are often referred to as supercomputers. The speed at which computing power increases is so fast that today's state of the art machines are obsolete after five years of operation on average. As the problems modelled in computer simulations and decision support systems grow in size and complexity (to enable more detailed predictions, to cope with ever larger amounts of data or both), so do the demands on computational resources. In many areas spanning from health, biology and climate change to automotive, aerospace energy and banking general-purpose computers cannot provide a practical solution to address complexity anymore and access to HPC becomes essential. HPC is at the core of major advances and innovations in the digital age, where to out-compute is to out-compete. It is a key technology for science, industry, and society at large. Annex 5 provides data on how impactful HPC is today on the economy and society, provides data on its high returns on investments, and illustrates the importance of HPC with many examples of the most prominent HPC applications in science, industry and the public sector. To fully reap the benefits of HPC, it is necessary to support a full ecosystem comprising hardware and software components, applications, skills, services and interconnections. In fact every new generation of HPC systems pushing further the performance limits is the result of a cooperative effort between suppliers, operators, users and researchers, tailor-made and optimised for the scientific or industrial application users, which HPC systems are intended to serve. Such an interdisciplinary approach leads to a more efficient use of often expensive supercomputing systems and develops training capabilities for programming such systems.

2.2.What is the problem that requires action and its size The 2012 Communication on "High-Performance Computing: Europe's place in a Global Race"1, laid down the foundations for a European HPC strategy. The overall objective was the development of the European HPC infrastructure and a pooling of national investments in HPC. This was broken down into the specific objectives to provide a world-class European HPC infrastructure, ensure independent access to HPC technologies, pool enlarged resources, increase efficiency, strategic use of joint and pre-commercial procurement, and ultimately, ensure the EU's position as a global actor. The Communication called on the European Commission, the Member States, the industry and PRACE to put in place a number of actions to reach these objectives. As a consequence the following actions were implemented: ·The contractual public-private-partnership (cPPP) European Technology Platform ETP4HPC was created in 2013. It is an industry-led technology platform of HPC suppliers with the mission to develop a joint research agenda and coordinate its implementation, thereby creating critical mass of R&D in HPC, building a world-class HPC technology supply chain in Europe, increasing the global share of European HPC technology vendors and maximising the benefits that HPC technology brings to the European HPC user community. ·The EC made a commitment towards the ETP4HPC to invest EUR 700 million in H2020 for the period 2014-2020. The cPPP should leverage a similar amount of resources on the private side. ·PRACE hosting member countries supported the European science community, providing access to most powerful computers in Europe, as well as training facilities and skills development opportunities. The EC supports financially PRACE to facilitate the access to the computing resources. In addition, the EC and the MS are also investing in GÉANT, the pan-European data network for the research and education community linking national research and education networks as well as supercomputing centres across Europe. ·Centres of Excellence (CoEs) were established and funded by the EC for the application of HPC in scientific domains of importance for Europe. They also provide support, competences and training to the user communities. ·MS continued to invest in their national HPC capabilities, developing technology and regularly procuring new machines to replace the outdated ones. Lessons learnt in the implementation of the HPC strategy Despite all the above activities that were put in place, the implementation of the HPC strategy is still not very efficient and effective today. The main reason for this is the existing fragmentation of efforts at EU programme level and between EU and MS level. As reported in the 2016 Communication on the European Cloud Initiative2 and in the accompanying Staff Working Document3, many of the challenges identified in the Communication of 2012 are still largely unresolved: ·Provide a world-class European HPC infrastructure: PRACE is an effective tool to provide computing cycles, but the procurement of systems is still done by MS in an uncoordinated way. This situation for example leads to periods of abundance of top-class systems (i.e. in 2012, with several MS acquiring top machines) followed by a period of low resources for researchers (i.e. in 2016-2017, due to obsolescence of previous machines and lack of further investments). A pan-European vision with a more strategic and rational planning of procurements is necessary. ·Fragmentation of European and national efforts: The implementation of the R&I agenda proposed by the cPPP is fragmented: First, MS have their own national programmes. And second, the way the EC implements the HPC strategy is not well-coordinated, since it uses two different programmes (CEF and H2020) and three different H2020 WPs (FET, LEIT-ICT and Research Infrastructures). This is due to the different nature of the supported activities (R&I, infrastructure development, etc.) and annual budget limitations. Such implementation is complex: it involves discussion with four different programme committees with different delegates often from different entities within each MS, making coordination between committees more difficult. Moreover, as the four different programmes have their own timing, the synchronisation of the calls addressing the various but interlinked aspects of the European HPC strategy, becomes a real challenge. At EU level, a single R&I programme is necessary for an efficient production of European HPC technology. ·Innovation procurement is not used in HPC: Innovation procurement instruments like the Pre-Commercial Procurement (PCP) and the Public Procurement for Innovation (PPI) have not been used so far by the MS in the area of HPC. In contrast, the USA, China and Japan use legal instruments that ensure a flexible process for the production of national technology in R&D programmes and their integration in the systems that are acquired by the national agencies. Europe would benefit from a joint structure that would permit to pool national and EU resources and jointly procure HPC systems by making systematic recourse to innovation procurement. ·Ensure the EU's position as a global actor: European suppliers face limitations in acceding to public procurements of HPC in USA, China or Japan. In contrast the EU is still the most open market, with no restriction in most of the public procurements on HPC (except e.g. for military purpose machines in some countries). Making systematically recourse to mechanisms like the one provided by Article 30.3 of the H2020 model grant agreement (to object under certain conditions to the transfer of Intellectual Property Rights (IPR) to third countries) and to new procurement and exploitation strategies would permit protecting IPRs produced in the EU and first exploiting in Europe the EU-funded R&I results. The above issues will be analysed in the sections below. In summary, the EU is now confronted with the following situation: The EU does not have the best supercomputers in the world even in areas of key importance; the available supercomputers do not satisfy the demand; the MS spending in HPC is not coordinated and the industrial take-up of HPC technology developments is low. We still fail in turning the technology development into HPC systems that are procured in Europe, i.e. we lack an effective link between technology supply, co-design with users, and a joint procurement of systems. Figure 1 provides an overview of the main problems, their drivers and their consequences. Figure 1: Overview of the main problems, their drivers and consequences

Problem Nr 1: The EU does not have the best supercomputers in the world but, in addition it is largely dependent on non-European HPC supply chains with the increasing risk of not having access to latest strategic technology even if resources were available. Today, none of the 10 leading supercomputers in the world – i.e. supercomputers with a performance level necessary to sustain leading-edge research – is located in the EU (5 are in the USA, 2 in Japan, 1 in Switzerland and the 2 top systems are located in China). Figure 2: HPC World Top 20 as of June 2017 and their computing power Figure 3: HPC World Top 20 as of June 2017 Europe's top performing machine is located in Switzerland and it is based on US technology (i.e. Cray). A more detailed analysis shows that while the US and the EU machines have similar capacities in terms of number of available computer cores, the performance of the two systems is very different with the average EU being below 30% of the average US performance. The main reason for this situation is that Collectively, the EU and the MS are significantly under-investing in HPC technology supply and infrastructures when compared to the investments of USA, China or Japan. The current funding gap with the USA is estimated in at least EUR 700 million per year. So far, the EC invested EUR 330 million on HPC-related activities of H2020 between 2014 and 2017 and further investments of the order of EUR 750 million are foreseen in the period 2018 – 2020. The total EU support in HPC will be over EUR one billion in the period 2014-2020. As for the collective investments of the MS for the same period these are estimated to be at around EUR 1.5 billion. The four PRACE countries hosting the Tier-0 machines made available to European scientists of computing time equivalent to the cost of EUR 400 million. A study conducted for the EC in 2015 on the progress of the implementation of the European HPC strategy concluded that the present pace of growth of European investment in HPC will not be enough to attain and maintain leadership, meaning at minimum parity with best-in-class HPC resources in the USA, Japan, or China. In 2015, the estimations of the study of the public and private investments for Europe to achieve leadership by 2020 were in the order of additional EUR 5.3 billion in 7 years (2016 to 2022). When compared to current investments of the EU and MS, the gap with the USA can be estimated at least at EUR 700 million per year (see Annexes 5 and 6 for further details). Moreover, HPC has now become an indispensable technology for supporting policy making and maintaining national sovereignty, supporting strategic decision-making for energy, home security, or climate change, or in the context of national security applications. Access to indigenous world-class HPC machines has become an asset to a country to an extent that it is considered not only as a strategic resource for its economy, but also a matter of national security. A recent report from the US states "… national security requires the best computing available, and loss of leadership in HPC will severely compromise our national security …". Access to own higher computing power provides a competitive advantage in scientific innovation and industrial competitiveness, and is indispensable technology for supporting policy making and maintaining national sovereignty (see Annex 5 for a detailed analysis of the impact of HPC). The USA, China and Japan have declared HPC to be a strategic priority for their country. They consider HPC as too strategic to be largely dependent on foreign suppliers and put focus on developing indigenous HPC supply chains and ecosystems that are essential for economic development and for security and safety. The top 5 supercomputers installed in the EU are supplied by vendors outside Europe, 3 from the United States (Cray, HPE) and 2 from China (Lenovo). They all integrate Intel processor technologies. Similarly, the technology used to interconnect the system originates from outside Europe (Cray, Intel, Mellanox). If the EU does not have its indigenous supply of supercomputers, it will find difficulties to acquire state-of-the-art machines or the technology to build them, as the supplying regions would not want to lose their competitive advantage to innovate. A recent example is China, which was deprived from the supply of USA state-of-the-art technology and as a consequence developed its own HPC supply chain. As a result China now has the two world-wide fastest supercomputers. The concern that Europe's supercomputing capabilities depend on non-EU suppliers for critical technologies and systems and that Europe is relegated against its global competitors in a field as strategic as supercomputing have been put forward by stakeholders, and in particular by the user industry, in the EuroHPC targeted consultation (see Annex 2) to go for a new action at the EU level that goes beyond current actions. EU depends on other regions for the supply of critical technology for its HPC infrastructure. EU risks getting technologically deprived of strategic know-how for innovation and competitiveness. The availability of the best supercomputing capacity is strongly linked to the ability to master and produce indigenous HPC technology in Europe. If the EU is not able to produce and integrate leading-edge HPC technology, it will systematically become a mere buyer of foreign systems that are behind the latest HPC technological generation that is produced, used and exploited first elsewhere. Therefore, it is of strategic importance for the EU to be able to produce its own HPC technologies and integrate them into leading-edge HPC machines that it procures.

In order to fully reap the benefits of leading-edge HPC machines, it is also necessary to create a full HPC ecosystem. Every new generation of machines pushing further the limits of performance is custom designed. These machines are not off-the-shelf products. On the contrary, they are a product of the cooperative effort of the suppliers, operators and users, tailor-made and optimised for the class of applications they are intended to solve and taking into account the boundary conditions (e.g. availability of affordable electricity). This last aspect is crucial to develop a sustainable HPC environment. To reap the benefits of the use of a supercomputer requires ownership of the infrastructure. In addition, it is necessary to coordinate the supply of hardware and software components and tailor it to meet scientific and industrial application requirements. This means that such diverse communities as low-power microprocessors designers, resource management software developers, system integrators and computational scientists and engineers have to work together from an early stage of the design and development phases of supercomputing systems. Such an interdisciplinary approach leads to a more efficient use of often expensive supercomputing systems and calls for developing training capabilities for programming such computers for different user communities. Problem Nr 2: Supercomputers available in Europe do not satisfy the demand Because of the lack of highest performing HPC machines available in the EU, the European scientific and engineering research community prefers to use USA supercomputing facilities rather than PRACE. Two US programmes provide such opportunity. Comparing PRACE with its overseas counterpart in the USA (the Advanced Scientific Computing Research (ASCR) programme) shows the extent to which the European HPC offer is insufficient to satisfy the demand. Access to ASCR supercomputer facilities is open to the scientific and engineering research community including universities, industry, and the US national labs through a peer review process. Data collected for the Fiscal Year 2015 reveals that the US is the major user of ASCR facilities with a total of 25993 users, whilst Europe with 3501 users (UK, Germany, France and Italy in the lead) is their second major user. Not only Europe does not have the best machines but it also cannot sufficiently satisfy the demand. If we compare ASCR and PRACE by the number of awarded projects by both programmes, we can see that the total number of projects awarded by PRACE in each call (PRACE has 2 calls per year) has never exceeded 60 projects, while solely in the 2016 Fiscal Year ASCR awarded 921 projects to entities from the European countries. This comparison is indicating that the European scientific and engineering research community obtain more support from the US ASCR supercomputing facilities than from the European PRACE. A striking fact is that even the four PRACE members (Germany, France, Spain, Italy) that are providing computer systems to the PRACE association obtained more projects from ASCR than the maximum PRACE could offer. This holds also for some associated members of PRACE like the Czech Republic, Poland, Denmark, and the UK that do not provide computing systems to PRACE. In conclusion, there is a strong demand for HPC access in the EU, a demand which is not sufficiently satisfied by PRACE. Insufficient access to HPC resources was also among the main reasons why respondents to the EuroHPC consultation indicated that there is currently a problem with HPC in Europe (for details see Annex 2). Regarding specifically the role of existing EU-funded HPC actions, such as PRACE, ETP4HPC and GEANT, they confirmed that in a future EU initiative on HPC existing actions should be improved, especially PRACE, and collaborate closer to complement each other. While we cannot take it for granted that Europeans can always use the best supercomputers existing in other regions, it is also a fact that the most demanding European applications have to run on the machines provided by the EU competitors, while only less demanding applications can run on the supercomputers available in Europe. This means that data produced by EU research and industry risks getting processed elsewhere for lack of corresponding capabilities in the EU. Ultimately our scientific and industrial leadership will become dependent on the accessibility to the highest-end machines that are outside Europe. The high dependence on the access to non-European supercomputers raises several problems: ·The data produced by European scientists and industry is processed outside the EU. This creates problems related to privacy, data protection, commercial trade secrets, and ultimately loss of ownership of critical data. This is particularly critical for sensitive applications, for example in security, health, or engineering, where the data for national security reasons should not leave the EU. ·European users do not have the priority to use the machines and are at the mercy of the access policy of the hosting country. Even if the selection of users is based on a peer review process, European users always face the risk that indigenous users get preferred access, computing time or computing power. This is in particular a problem for industrial users that cannot afford to wait for the machines they need. ·In the long term, European users, scientific as well as industrial, might move outside the EU to get the same access rights, access conditions and price as the indigenous users. The brain drain from researchers relocating to third countries may not be limited to the loss of excellence in scientific disciplines, but Europe can also lose its competences in developing applications for supercomputers if users no longer co-design the applications with the supercomputing centres. ·The demand for more computing power will continue increasing and new applications will soon emerge notably in health, energy, environment, fintech, manufacturing etc. which we cannot satisfy with the computing power of today's HPC machines. The problem will become even more acute in the next 5-7 years when the development of exascale applications comes to maturity. Problem Nr 3: Member States do not have a framework for joint procurement The PRACE Association provides access to the most powerful supercomputers in Europe (Tier-0 machines). However, it does not cover the coordination of national programmes, nor joint investments for the procurement of systems, e.g. there is no common European strategy to develop and acquire pre-exascale or exascale machines. The design, the specifications and the procurement of the machines are done by each of the supercomputing centres on their own and are mainly guided by own or by national interests, without any incentive to coordinate with the other countries. Although most MS share the same interests in advancing science, they try to satisfy as much as possible the requirement of their national scientific communities. This however does not guarantee an optimal coverage of the different scientific communities at European level. In the USA, China and Japan the high-end supercomputing resources are acquired through public procurement of innovation with a national strategy. For example in the USA, there is a federal coordination of procurements between different agencies, like the National Strategic Computing Initiative (NSCI) and the Exascale Computing Project (see Annex 6) to frame the national efforts in a coherent strategy. These initiatives provide a critical mass for procurement, obtaining better value for money in acquisitions, and are also tightly linked to the technology supply, ensuring that national suppliers stay at the forefront of technology advancements. In Europe, the large fragmentation of HPC programmes and efforts, the non-coordinated activities and the lack of a common procurement framework lead to a waste of resources. This has been identified by stakeholders in the EuroHPC consultation as one of the current two top ranked problems of HPC in Europe (see Annex 2), independent of their type of organisation (i.e. scientific user, industrial user, technology supply industry, computing centre).

Current funding instruments have limitations when applied to large mission-oriented initiatives. The existing implementation tools are well adapted mainly to support R&D of marketable HPC technologies, but are complicated to coordinate for example several synchronised procurements in different MS with different legislation and rules. Most MS and Associated Countries to Horizon 2020 have their national HPC strategies and investment plans according to their national needs and programmes. In particular those countries that have not the capability to invest in their own leading-class machines are prepared to co-invest to get access to the Tier-0 machines. Most of them participate in the PRACE programme. Europe thus misses the opportunity to take advantage of efficiency gains by aligning the strategies and pooling resources. As for the EC, so far it has provided limited support to a joint procurement of supercomputers, mainly because of the hesitation of the Tier-0 countries to participate in such joint procurement. In 2016, the EC put in place a first call for procurement of innovation with the PRACE hosting countries. In return of the financial support from H2020 (at the level of 35% of the overall costs), the participating countries agreed to provide access to the procured machines. Problem Nr 4: The European HPC technology supply chain is weak and the integration of European technologies into operational HPC machines remains insignificant Today, Europe consumes about 29% of HPC resources worldwide, but the EU industry provides only ~5% of such resources worldwide. In addition, close to one fifth of the top 500 HPC systems are located in the EU, and out of these, ~20% are provided by EU vendors (oscillating between 20% and 25% over the last years). Between 2015 and 2016, their exports outside the EU consisted of 3 systems installed in South America and 5 in Asia. Figure 4: Contribution of EU vendors to the 500 most performant HPC systems (2011-16) The market share of EU vendors in Europe is even smaller when considering the HPC server market (beyond the top 500 systems), although the growth rates have been largely beating the market average (15% vs 3.4%). This growth raised the EU suppliers' market share from 2.8% in 2013 to 3.7% in 2016. During the same period, EU suppliers' share of the global HPC server market expanded at a robust 16.1% annual growth rate. Although this growth substantially exceeded the market's 5.8% growth rate during this period, it enlarged EU suppliers' share of the global HPC server market merely from a marginal 0.8% in 2013 to an equally marginal 1.1% in 2016. On the global market the European suppliers face unequal treatment on public procurement. The USA and China restrict the development and procurement of the high-end machines to domestic suppliers. As a consequence, non-European suppliers have a clear competitive advantage as they get direct funding support for the development of the national machines that they later on sell on the global market. In Europe there is an open market, without a policy to favour European suppliers. Therefore, in the absence of a prospective lead market and a risk sharing with the public sector, European suppliers hesitate to take the financial risk to develop the technology on their own. Despite the national and European R&D funding programmes, Europe ends up with a weak supply industry, while it has one third of the application markets. As the non-European HPC suppliers participate in the European R&D programmes, the EU ends-up paying non-EU-suppliers twice: for the development of their underlying technology and for the acquisition of their HPC machines built with non-EU technology. It is also to be noted that the functional HPC components and prototypes developed by the projects funded through the European Framework Programmes for Research (FP7, H2020) are rarely integrated in the machines that are procured. There are three main reasons for that: ·Firstly, there is a lack of incentives of the supercomputing centres that specify the machines to be procured to privilege solutions developed by the European R&D programmes. The procurements are in general implemented by the MS according to their national rules and the EC cannot impose measures to favour European suppliers. ·Secondly, the H2020 rules for participation make it difficult to ensure continuity of the investments made between different calls. Each call is an open and competitive process, with the possibility to limit a follow-up call to the successful projects of the previous call only in exceptional cases. This leads to the situation where R&D results are rarely integrated in the subsequent calls addressing the development phase of the supercomputer, and are replaced by a solution that was developed elsewhere, including from a non-European supplier. The R&D investments are then inefficient to transfer European R&D results into marketable products, or to foster a European supply industry. European companies have then an increased risk of not getting funded in the subsequent call. The problem is further exacerbated by the fact that the H2020 calls are open also to non-EU beneficiaries. ·Thirdly, the EC (and MS) do not use the innovation procurement instruments (PCP combined with PPI) to accompany the route from HPC technology development to procurement that would help support a competitive European supply industry and create a lead market in Europe. The main reason so far is the hesitation of the Tier-0 countries to participate in joint procurement actions, even less to ones that may favour procurement of European technologies that may be perceived as not sufficiently competitive. Finally, the development of exascale technologies is not for the sake of having the fastest supercomputer in the world, but the goal is to build "first of a kind" systems rather than "one of a kind". Indeed, HPC technology of today, and in particular low-power processing units and systems, is the mainstream technology that we will find in the next five years integrated in our cars, homes, factories and personal devices. Not investing in HPC technologies makes it difficult to be present in any digital technology in the future, like the autonomous vehicles, the connected car, or the smart home. The transition to exascale computing is an opportunity for the European supply industry to leverage on technologies in the computing continuum from smart phones, to embedded systems (for example in the future driverless cars), and to servers, feeding the broader ICT market within a few years of their introduction in high-end HPC – giving a competitive advantage to those developing them at an early stage. The size of these target markets is estimated to EUR 1 trillion. The industrial users who responded to the public consultation identified as their main concern the dependence on non-EU technology. This clearly shows their awareness of the risks related to the dependence of a foreign technology supply-chain for a resource that is a critical for their competitiveness on a global market.

2.3.What are the problem drivers? The following are the main drivers contributing to the problem: Problem Driver Nr 1: Public funding for HPC in EU/MS remains uncoordinated and insufficient to cope with the demand MS investments are insufficient and uncoordinated to acquire enough high end HPC systems that satisfy the demand. According to the 2014 International Data Cooperation study19, Europe started to narrow down the former wide gap separating the most capable US and Japanese supercomputers at the very high end of the supercomputers segment from their European counterparts. Indeed, at first, spending increased substantially in the EU for large supercomputers from 2009 (112 million EUR) until 2012 (658 million EUR). However, in 2014, it started to decline again (362 million EUR in 2014). No MS has the capabilities to develop the necessary HPC ecosystem on its own in a competitive timeframe with respect to the USA, China or Japan. The individual MS do not have the full value chain or competences and most lack the necessary funding levels. Lack of sufficient resources is one of the main reasons put forward in the EuroHPC consultation (see Annex 2) to go for a new action at EU level. The leading regions in the world are racing ahead and are massively investing in strategic HPC programmes to boost their HPC ecosystem and prepare it for the upcoming next generation computing (exascale and beyond). These programmes are driven by the public authorities with some leverage of private investments (mainly on the technology supply side). If the EU is entirely dependent on non-European supply, this puts at risk our capacity to acquire the latest HPC systems and our capacity to build a digital industry all together given that HPC technology has a spill over effect on all digital technologies. Regaining world-wide leadership in HPC cannot be achieved on the basis of Europe's current HPC industry set-up and market conditions alone. No European industrial player currently has all competences in-house. The required investments levels for industry exceed their capacity and the risks of failure to develop an exascale system are too high to be borne by industry alone. Equally, public funding alone will not be sufficient to finance the broad uptake of HPC in the European industry in coming years and notably SMEs. Problem Driver Nr 2: European HPC system vendors face stiff competition from large foreign corporations Relatively small European HPC system vendors face stiff competition from large foreign corporations supported by their governments on the open European HPC market. European HPC vendors face asymmetries in major HPC markets outside of Europe due to national regulations e.g. for national security. The concern that the EU is relegated against its global competitors in a field as strategic as supercomputing is among the main reasons for respondents to the consultation for a new action at EU level that goes beyond current actions (see Annex 2). Other countries such the USA have long-standing models for R&D collaborations with indigenous HPC vendors, many of which include supercomputer procurements with strong R&D requirements. Building an HPC ecosystem is a significant challenge, because the EU has historically been the most open major HPC market in the world, in part because it has not had an indigenous HPC system vendor large enough to compete with US, Japanese or Chinese vendors. As a result, in 2014, 81.2% of all European HPC server system spending profited US vendors. The only sizeable Europe-based vendor, Atos-Bull, accounted for only ~2% of European HPC server spending in 2014. It has had some successes outside of Europe, but still relies on European sales for a large majority of its HPC revenues. This unsatisfying situation is exacerbated by three factors: I.European HPC system vendors face asymmetries in major HPC markets outside the EU due to national regulations, e.g. related to national security. II.The EU market for HPC hardware systems and parallel software is still too small and fragmented to support EU-based HPC vendors. They cannot thrive and continually fund world-class innovation unless they can match the investments of competitors (especially USA and China) that have strong domestic demand and easier access to the global market. III.Intellectual property rights developed in EU research projects relevant to HPC often benefit non-EU parents of participating companies as the current EU Framework Programme for Research and Innovation imposes limited restrictions on the transfer of rights to affiliates in third countries. Concerning the 1st factor, in the USA the acquisitions of supercomputers by US federal agencies are restricted by the "Buy American" Act, although purchasing of software and components of non-US origin is often allowed. In China, the fast-growing HPC market has been dominated historically by US supercomputer vendors, because Chinese HPC vendors have not been able to compete effectively. More recently, the Chinese government directed investment banks and other "critical infrastructure" sites to cease acquiring non-Chinese HPC systems. In Japan, the government market has historically tended to favour Japanese supercomputers, although non-Japanese ones also had some success in this market. Concerning the 2nd factor, the EU has a lower aggregate level of HPC resources compared to other large economies around the world. With regard to the top 500 machines, the USA have ~50% more HPC resources than the EU, and China and Japan together have approximately twice the resources of the EU. These differences are significant as these three global blocs have comparable nominal economic GDP outputs. In other words, there is currently indeed a structural weakness in EU HPC resources.

Concerning the 3rd factor, there are provisions in the EU Research Framework Programme that can contractually oblige an organisation to disclose such transfers. These restrictions are certainly helpful although not as severe as in other countries developing HPC technologies outside the EU. In addition to issues related to IPRs and their transfers, probably an even more important aspect is that a stronger HPC ecosystem in the EU is likely to open up new and attractive career opportunities for top scientists and engineers and reduce the brain drain from the EU. 2.4.Who is affected and in what ways? A joint structure coordinating and pooling the resources at European level will mobilise the necessary resources at European level to provide a world-class pan-European infrastructure and a strong European HPC ecosystem with lasting benefits in Europe. Europe in principle has the human potential and technological know-how to develop such an ecosystem along the whole HPC value chain. The following group of stakeholders are likely to be directly or indirectly affected by the initiative (see Annex 3 for an extended analysis). Member States MS are expected to significantly benefit from the initiative. The EuroHPC initiative will enable them to coordinate together with the EC their HPC investments and strategies. The joint initiative will make it possible for them to access a world-class HPC infrastructure that no single country on its own can afford in particular those with little or no significant HPC resources in place. The increased availability and accessibility of top HPC resources will motivate the users to keep their activities and data in Europe, helping to keep critical know-how and human potential in MS. Science - Universities and Research Centres Thanks to a joint structure that ensures the sustainability and availability of resources in the short, medium and long terms, EuroHPC will ensure a European-wide access for researchers in Europe's Universities and Research Centres to supercomputers and data with a guaranteed high level of resources, and irrespective of their geographical location or scientific discipline. This factor is critical to ensure that the academic and scientific potential stays in Europe and is not exploited in other regions with more competitive HPC resources. A sustainable joint structure supported by the EC and MS will also consolidate the already existing vibrant mix of national, regional and pan-European initiatives in intra-EU scientific collaboration, and will provide EU-based teams with powerful resources to strengthen the European presence in international scientific endeavours. Industry including SMEs The new initiative will revitalise the European HPC ecosystem, where industry and in particular SMEs will benefit as both users and suppliers of HPC technology and applications. ·As users; EuroHPC will consolidate European leadership in many HPC-empowered applications by making available more resources for industrial use accessible at EU level, complemented with specific measures to widen the usage of HPC technologies. This is of critical importance to industry and particularly SMEs without in-house capabilities that will benefit from easy to use HPC resources, applications and analytics tools to create new innovative products and processes. ·As suppliers; a European-wide initiative with a focus on the supply of a European source of HPC technology and the protection of European IPR will have the necessary critical mass and a catalytic effect on the European suppliers. A clear R&I roadmap at European level provides a unique opportunity for industry, including SMEs, to participate in the co-design and development of such new technologies and systems, and to develop IPR and solutions to be further used in their business endeavours. European Commission (EC) The EuroHPC initiative would solve the current complexity of implementing the HPC activities through two different programmes (CEF and H2020) and three different H2020 WPs (FET, LEIT-ICT and Research Infrastructures). The EuroHPC entity will provide a single structure and financial framework to coordinate the different activities in synergy. Most importantly, it will provide a single forum for strategic discussions with the MS and leverage EU and national efforts and resources. The EuroHPC entity will also be a privileged interlocutor for institutions, agencies and bodies addressing critical scientific, industrial or social-impact areas. It will become a focal point for better supporting the EU policy development and implementation in areas like digitising industry (DSM), security, and many other related to societal challenges. Supercomputing Centres The EuroHPC entity will provide the appropriate frame to strategically plan for the further development and the federation of supercomputing centres at European, national and regional level. The joint structure will avoid redundancies and will exploit complementarities with a European-wide planning of the different architectures across Europe (for example avoiding isolated and uncoordinated procurements that may end up in dependencies on single vendors and technological suppliers). EuroHPC also provides the opportunity to fully exploit the world-class HPC infrastructure and human resources of the European supercomputing centres in a synergetic way, encompassing the co-design and integration of technology with a coordinated procurement of supercomputers at European level.

Citizens EuroHPC will ensure that world-class HPC resources and data are available for applications that are of direct interest for citizens. Given the inter-disciplinary nature of HPC and the wide range of scientific and industrial applications that will be made available at EU level, citizens will benefit from an increased level of resources provided by EuroHPC in areas like: ·Health, demographic change and wellbeing ·Secure, clean and efficient energy ·Smart, green and integrated urban planning ·Cybersecurity ·Weather forecasting and Climate change ·Food security Third Country Actors Successfully building a European HPC ecosystem will have an effect on the non-EU supply industry. The focus on the new instrument to produce, co-design and take-up of European technology in the next generations of European supercomputers will make EU technology more competitive. This will eventually decrease the market share of non-EU HPC components and systems in Europe, potentially worldwide. The increased protection of European IPR resulting from the R&I programmes supported by the EuroHPC, may stop the current situation of non-EU suppliers taking advantage of EU programmes to export the resulting IPR and improving their domestic developments. Provided access conditions on equal terms becomes a global practice, the European HPC resources could become attractive for scientists from outside the EU, sending their data for processing to Europe. The risk Europe currently faces with losing its data sovereignty may thus be reversed. 2.5.How would the problem evolve? There is an arms-race world-wide to develop and operate ever more powerful supercomputers. This is driven by the strategic importance for a nation and an economy for top-level computational power, but also driven by the growing demands of the scientists to solve ever more complex problems. The renewal of a machine every 5 years at the end of its lifetime increases the costs for the development, installation and operation of the machine by a factor two. The costs have now grown to an extent that they have become prohibitive for most market actors, including for most national governments in Europe. The effect has started to show as Europe is slowly dropping out of the first league of supercomputers. After a height of four machines ranking in the top 10 most performing supercomputers worldwide in 2012, the number has been steadily decreasing since, until EU based machines dropped out altogether from the top 10 list in 2017, despite recent acquisitions/upgrades at several sites across the EU. Without an increased effort to invest more or more efficiently, for example by pooling resources in Europe and coordinating acquisition planning, this trend will continue and accelerate. According to the IDC study on High Performance Computing in the EU: Progress on the Implementation of the European HPC Strategy19 "… High-end supercomputers at national centres in Europe, the U.S., and Japan (but not China) are regularly oversubscribed – the demand for computing cycles typically exceeds the supply by a factor of two to three. European high-end supercomputers are no exception…". The demand for HPC is expected to increase considerably in the coming years, for example as more and more users, in particular SMEs, become part of the digital economy and make use of HPC for their business processes. This is paired with a hunger for higher computing power as the applications increase in complexity and include big data analysis. The gap between demand and supply of the most performant computing facilities will thus increase. A growing gap of available computing performance will motivate more and more European scientists to implement their calculations outside Europe. Europe thus runs the risk of losing control of its scientific data, but eventually also the brain drain of its scientists as they move closer to the computing facilities. The situation is similar for the industrial users who will not renew their service contracts with European supercomputing centres, but replace them with contracts outside the EU in order to stay competitive on a global market. The move of the users outside Europe would have a cascading effect on the supercomputing skills in Europe. Indeed, the supercomputing centres work closely with the users, equipment suppliers and program developers to adapt the applications to the architecture of the supercomputers and optimize the use of computing resources. This co-design approach is essential for an effective exploitation of the supercomputers and requires a profound understanding of the machine's architecture and behaviour. Breaking this co-design chain will gradually erode the skills of the supercomputing centres to offer competitive computing services as they would not be involved in the design of the most powerful machines. Ultimately, Europe runs the risk to lack the competences to design and operate exascale machines and offer computing services to the most demanding applications. The European suppliers with the competences and financial resources to provide the European market with the required machines, without public intervention hesitate to take the risk to invest in this field. Indeed, the market size of high-end machines is very small, maximum 100 machines per year worldwide but mostly in very closed and inaccessible markets for European vendors. The situation is identical for the European competitors in the USA, China and Japan. Those countries have already declared HPC to be a strategic priority for their country. As a consequence their governments fund programmes to develop national HPC ecosystems and work on the deployment of exascale supercomputers. As a consequence USA vendors (with new Chinese players) will continue to dominate the market. If no effective measures are taken to bring the EU back in the HPC race with EU technology, the dependency of Europe on non-EU suppliers will continue to grow. Otherwise the industrial base of European suppliers will continue to decrease as the companies either stop their HPC products (e.g. Eurotech), or do not find sufficient market share to survive. As a consequence the European HPC ecosystem will further erode.

The dominant suppliers like Intel, due to their market share, are dictating the prices and are gradually pushing European competitors out of the market. Initially those suppliers were in essence supplying the cores of the supercomputers, giving European suppliers the possibility to provide the other components (e.g. interconnects) and integrate them in the computing boards. Gradually the dominant suppliers started to provide the other components too, proposing complete solutions. The procurers of the machines on one hand have less and less the choice to include components from European suppliers as the solutions offered by dominant suppliers provide less and less integration possibilities. In the absence of a European supplier of the computing cores, the procurers are becoming more and more dependent on complete packages, limiting their possibility to co-design the machines according to their needs. Moreover, there is a risk that due to certain strategic or political decisions, such as export restrictions, sooner or later Europe will not have access to the most competitive and innovative technology, exacerbating the gaps described above. Therefore, to mediate the risks, the European HPC strategy has identified an action to develop one of the critical components of the exascale machines, i.e. the multipurpose low power processor. 3.Why should the EU act? 3.1.Why do we need to act now? The problems and risks detailed in the previous section require urgent action at European level. Our world competitors massively fund programmes to develop national HPC ecosystems and work on the deployment of exascale supercomputers, with a particular focus on the development of indigenous HPC supply chains that are essential for economic development and for security and safety. To stay competitive in the HPC race Europe needs to acquire exascale supercomputers by 2022/23 and develop its technology supply chain that guarantees access to latest HPC systems. To reach the target the work has to start now, since a development cycle typically takes 5 years. USA, China and Japan have also set 2022/23 as a target date. From the drawing board to the operational machine, the different components have to be developed and then integrated into the machine. An intermediate, key milestone is the development of the pre-exascale machines by 2020, where the initial design will be validated and the prospects of reaching the exascale target will be assessed. Any delay in the acquisition of the pre-exascale would equally delay the development of the exascale targets. Without more investments in the years 2018-2020 there is the risk that the pre-exascale target cannot be met, jeopardizing the longer term objective of the exascale machines. The EU funds to acquire the pre-exascale machines and develop its ecosystem are available now (EUR 486 million, already committed in H2020 and CEF WPs 2019-2020 to HPC activities). There is also evidence from several MS/AC (at least those which signed the EuroHPC declaration) that they are prepared to commit similar funding levels in the same period to implement the joint EuroHPC activities. Setting up a self-standing EuroHPC entity in 2019 will ensure that Europe takes action in a timely and coordinated way, with joint investments in leading-edge HPC technology and infrastructures. It would allow Europe to acquire world-class pre-exascale machines and stay competitive in leading scientific and industrial applications for the period 2020 to 2025; and, equally important, to build in Europe the necessary ecosystem to develop competitive European technology for the pre- and full exascale computing era. In an eventual support of the EuroHPC entity under the next MFF, this would allow to continue its operations and in particular to: procure and deploy two world-class exascale machines (based on European-funded technology); support the development of ambitious extreme scale applications for public and private users and of HPC skills; and, continue to support R&I activities, notably a competitive low-power European microprocessor and post-exascale HPC machines and their linking to the first quantum computing infrastructures that would have been developed under the Quantum Technologies FET Flagship. 3.2.The right to act at EU level The fragmentation of public HPC services across the EU and within MS leads to inefficient use of resources and only partial cross-border exchange of expertise. The increasing costs of building and maintaining HPC infrastructures require stronger governance at EU level and the rationalisation of HPC resources to reduce the current fragmentation. HPC is an essential instrument to address societal challenges like health and security. Both are policies of shared European interest, as exemplified in the NIS Directive or the Cybersecurity Communication, addressing issues that do not stop at national borders. The level of security or the quality of public health in one MS depends from the situation in the rest of the EU. HPC is fundamental to build the data economy. Controlling how the data is used, who has the ownership and right for exploitation, where it is stored, and who has access to it are sensitive issues. It touches commercial and copyright issues, but also data protection and privacy issues. All these issues have been identified as political priorities in the Digital Single Market (DSM). Sending sensitive European data for processing in other regions of the world, where the high European standards of privacy, data protection, copyright, etc. are not necessarily respected, undermines the intention to gain sovereignty on European data and its exploitation. The scale of the resources that are needed to realise a sustainable exascale level HPC infrastructure and ecosystem is beyond what national governments can nowadays afford to invest. No single Member State has the financial means to acquire exascale computing capabilities and develop, acquire and operate the necessary exascale HPC ecosystem on its own and in competitive time frames with respect to the USA, China or Japan. Member States and national actors have now realised that they will only be able to remain competitive through a joint and coordinated EU-wide effort – c.f. the EuroHPC declaration of 23.03.2017. This justifies the right for the EU to act in the field of HPC under Article 179 that states that "The Union shall have the objective of strengthening its scientific and technological bases by achieving a European research area in which researchers, scientific knowledge and technology circulate freely, and encouraging it to become more competitive, including in its industry, while promoting all the research activities deemed necessary by virtue of other Chapters of the Treaties". In addition, Article 187 TFEU gives authorisation to "set up joint undertakings or any other structure necessary for the efficient execution of Union research, technological development and demonstration programmes". 3.3.Subsidiarity Knowledge and resources available in Europe need to be put together for the building of a leading edge HPC ecosystem across all value chain segments. National resources alone are insufficient. EU coordination of investments and resulting services is necessary in order to have HPC computing and data infrastructures as well as a full HPC ecosystem in Europe on a par with the USA, China or Japan. The importance of HPC for science, and the public and private sectors has grown in recent years together with the exponential rise in the investments required to stay globally competitive. This has led to a common understanding that the "Europeanisation" of this domain via a shared infrastructure and common use of existing capabilities would benefit everyone. This also applies to Member States with difficulties in creating self-sufficient national HPC infrastructures whereas they can make valuable contributions to and benefit from federated and interconnected EU-level HPC capabilities. Cooperation exists already in some areas among Member States, industry and science actors. Examples include PRACE, the HPC cPPP, the Big Data PPP and GÉANT. EuroHPC builds on them as the key investors in the EuroHPC signatory countries are already represented there.

There is however a need for a legal and financial framework closing the chain from R&D to the delivery and operation of the exascale HPC systems co-designed between users and suppliers. Such new framework would support the development of a pan-European HPC and data infrastructure built on European technologies, allowing the blending of different EU financing sources with national and private funding. It would stimulate joint investment to cover the ever increasing costs of building and maintaining world class HPC and data infrastructures and to optimise a number of important procurement factors, such as strategic planning for funding of top-level systems, user's coverage, diversity of systems architecture, etc. Ultimately, it would permit to build-up a critical mass in the HPC market to foster a true European HPC ecosystem that encourages and supports a competitive European HPC supply industry. As described in the introduction, political support from Member States on EuroHPC has already been explicitly given by the Council, by the signatories of the EuroHPC Declaration, and by the European Parliament. 3.4.EU added value The proposed initiative represents a natural candidate for European action with the EU and Member States implementing the EuroHPC strategy via the most appropriate legal instrument. It is only by acting at EU level that we will be able to pool together the necessary effort and create the critical mass needed to acquire leading next generation exascale systems which are in the order of hundreds of millions of Euros. Only by working at EU-level and combining investments, knowledge and skills Europe has a chance to keep-up to its competitors. At the same time, pooling the investments to jointly acquire exascale machines will create significantly higher return-on-investment (ROI) for each of the partial-owners of the machines, than the ROI of the full ownership of a lesser performing machine. Availability of top class HPC systems would enable European players to further develop a whole range of present and future scientific and industrial applications that would require exascale performance. It would permit funding the HPC related research and developing the necessary expertise, skills and capabilities for programming such systems efficiently and exploiting their full potential. And, most importantly, it will enable all European scientists, public administrations and industry to access this infrastructure and foster a wide range of cross-border collaboration and new products and services. Finally, by pooling the fragmented knowledge and the expertise existing all around, Europe can build the full supply chain for HPC systems: from technology components and systems to full machines. These are at the same time essential technologies in a variety of other mass markets (such as automotive, consumer electronics, servers, etc.). The transition to exascale computing, supported by joint EU/MS investments acting as lead market users, would provide an opportunity for the European supply industry to leverage on such investments and get access to new markets estimated to EUR 1 trillion. Overall, the creation of a globally competitive HPC environment in Europe, triggered by public intervention, creates goods and services that are of a truly public value for European science and industry: It will help the private and the public sectors to create leading-edge science, technologies and solutions benefiting all areas of the economy and society, contributing to the EU's objectives of economic growth, jobs and competitiveness. 4.Objectives: What should be achieved? 4.1.Overall objectives of the EuroHPC entity and instrument To address the problems identified in section 2, we propose to establish a new legal and financial instrument that would permit to implement the joint EuroHPC entity. Its overall mission would be to provide European scientists, industry and the public sector with the latest HPC and data infrastructure and support the development of its technologies and applications across a wide range of fields. The (EuroHPC) instrument would have to address the following three overall objectives: 1.Support a joint procurement framework for an integrated world-class exascale supercomputing and data infrastructure in Europe. Such infrastructure will be accessible on a non-economic basis to public and private users for publicly funded research purposes; 2.Ensure a EU level coordination and adequate financial resources to support the development, procurement and operation of such a public infrastructure; 3.Support the research and development of an integrated European HPC ecosystem, covering all scientific and industrial value chain segments (hardware, software, applications, services, interconnections and skills). 4.2.Functionalities of the EuroHPC instrument To reach its objectives, the EuroHPC instrument has to meet several functional requirements: [Pooling together public and private resources] The legal instrument would have to support the joint procurement and operation of a European HPC infrastructure as well as R&I activities for the development of European HPC technology and excellence in applications. To reach the necessary investment levels to meet its objectives, the instrument has to make possible the pooling of public funds (from the EU and the MS/AC), as well as financial or in-kind contributions from the private sector. The Union's financial contribution would come from the budgets already allocated to H2020 and CEF WPs in 2019 and 2020. Those funds would have to be matched by a similar amount from the MS and AC, as well as by the private sector. [Procuring HPC machines] The legal instrument should enable the EuroHPC entity to launch calls for tender to acquire the pre-exascale machines and select the successful tenderers. As EuroHPC is a joint initiative it would act as trusted manager of the procurement process on behalf of all participating countries. To reduce the administrative burden the legal instrument should permit the application of European procurement rules. Ideally, the legal instrument should allow an exemption from VAT to reduce the overall cost of the systems to be procured. [Openness to new partners] The legal instrument would have to allow new MS/AC to join the EuroHPC entity upon their request, subject to their financial commitment. EuroHPC could then start with a core group of participants that are ready at the launch of the instrument and gradually accept new members.

[Enabling participation of the private sector] for defining and implementing the EuroHPC strategic R&I and application development agenda, stimulating large private investments in the field… while mitigating any eventual conflicts of interest in public procurement processes: While enabling private participation would be fundamental, the EuroHPC instrument would also have to make possible to include provisions for avoiding conflicts of interest, notably by making sure that there is no interference of the participating technology supply industry neither in the joint public procurement process of the pre-exascale machines nor on how the public funds are spent. [Implementing a R&I programme which can address the present programme coordination inefficiencies in implementing the HPC strategy], in particular the difficult coordination and synchronisation of the different H2020 and CEF HPC WPs. This could be addressed by delegating the implementation of the related budget to the EuroHPC (legal) entity. Its governing body could then align the content and the respective timings of the different calls with the HPC strategic agenda; ensure coherence between the topics of the different calls; and put in place the appropriate funding instruments to reach the objectives, in particular innovation procurement for accompanying the route from European HPC technology development to the procurement of European machines. Furthermore, by using the H2020 rules, the EuroHPC instrument should make it possible to introduce provisions to protect the economic and strategic interests of the Union, i.e. protecting IPRs produced in the EU and first exploiting in Europe all EU-funded R&I results. [Safeguarding the Union's interest through EC participation] The EuroHPC instrument should foresee the possibility for the EU represented by the Commission to be part of its governance. That would ensure that the EC can play a significant role in the definition of the strategic orientation and priorities of the EuroHPC entity, and take part in the decisions on how its budget is allocated and spent. In essence, the legal instrument would have to permit a delegation of the EC funds to the legal entity, rather than a co-fund. [Lifetime] The EuroHPC instrument would have to exist until the termination of all its activities that would be launched in 2019 and 2020. Considering the typical duration of the H2020 grants and the typical lifetime of an HPC machine, the legal instrument would have to operate until approximately end 2026. To summarise, the EuroHPC instrument would have to address the following functionalities: Pool public and private funds Execute joint procurements while operating under EU-law (e.g. VAT exemption, procurement rules, …) Implement a R&I programme which can address the present coordination inefficiencies in implementing the HPC strategy Remain open to incorporate new MS/AC willing to join Enable participation of the private sector Safeguard the Union's interest (the Union will be represented by the Commission) 4.3.Specific Objectives Considering the general objectives of the previous section within the broader context of the European HPC Strategy, the EuroHPC entity and its related instrument should achieve the following specific objectives, grouped in three pillars. The overall EuroHPC objectives can only be achieved if all the three pillars are implemented, as each of them is necessary to create the European HPC ecosystem.







Pillar 1: Infrastructure development, acquisition and operations T1 Procurement and operation of world-class HPC and data infrastructures for European use, in particular aiming at procuring and operating two pre-exascale HPC machines (2020). T2 Interconnecting regional, national and European HPC resources (pre-exascale machines, data centres and associated software and applications) through an HPC and Big Data service infrastructure facility. T3 Providing access to HPC-based infrastructures and services to a wide range of users (scientific and industrial users including SMEs, and the public sector) for new and emerging data and compute-intensive applications and services. Pillar 2: Applications and skills development and a wide use of HPC T4 Achieving excellence in HPC applications through, for example, development and optimisation of codes and applications in a co-design approach, support to Centres of Excellence in HPC applications, and large-scale HPC-enabled pilot demonstrators and test-beds for big data applications and services in a wide range of scientific and industrial areas. T5 Increasing the innovation potential of industry, and in particular of SMEs, using advanced HPC infrastructures and services, for example via dedicated digital innovation hubs. T6 Outreach and training actions for attracting human resources to HPC and increasing skills and engineering know-how of the EuroHPC ecosystem.



Pillar 3: Research & Innovation agenda for European technology & know-how development T7 Developing the next generation of key HPC technologies and systems towards exascale addressing the whole European technology spectrum from low-power microprocessors and related technologies to software, algorithms, programming models and tools, to novel architectures and their system integration through a co-design approach. 5.Available Policy Options 5.1.Option 0: Baseline from which the policy options are assessed The HPC strategy is implemented at EU level mainly via H2020 and CEF: ·The development of the European HPC technology is addressed in the H2020 FET and LEIT-ICT WPs, from the research of the underlying concepts up to their integration; including the co-design and prototyping of HPC machines. The EC is supported by the cPPP (ETP4HPC) which submits every year its strategic research agenda (SRA) to the EC. The SRA provides the long term planning of activities necessary each year to reach the objectives of building a world-class HPC technology supply chain in Europe, including an estimate of the required budgets. Based on this input the EC then drafts the relevant work-programmes. ·The development of the HPC infrastructure is addressed in the H2020 Research Infrastructures and CEF WPs. The EC provides funding to facilitate access of European scientists to the Tier-0 machines operated by PRACE. This includes financial support to the additional services PRACE provides to the user communities, complementing the work of the HPC Centres of Excellence (CoE).14 While PRACE is an organisation independent from the EC, the CoEs are a creation of the EC and are almost exclusively funded from the H2020 budget. Due to the different nature of the HPC activities funded by the EC, their difference in scope, and annual budget limitations, the activities are implemented in four different WPs: FET, LEIT-ICT, Research Infrastructures and CEF, each with its own selection process and timing. The EC publishes open and competitive calls for proposals. Any beneficiary eligible for H2020 funding can apply, irrespective of whether it is member of the HPC cPPP. For the period 2018-2020, around EUR 110 million have been committed for HPC activities in CEF and another EUR 640 million in H2020. The R&D projects funded through the H2020 calls are generally reaching their objectives (as this is acknowledged by the project reviews carried out by independent experts) and contribute to establishing the European HPC ecosystem. However, as mentioned in section 2.2, today the EC has difficulties in implementing coherently and effectively the HPC research roadmap because its activities are funded through four different WPs of H2020 and CEF; and there is a lack of continuity in supporting research teams from R&D components to systems development and integration and ultimately to innovation. The current implementation has also weaknesses with respect to the protection of the IPRs. The H2020 calls are open to any legally established legal entity, based in a MS or third country. As non-European entities can participate in H2020 grants, either directly, or through their European subsidiaries, R&I results are often transferred and exploited out of the EU. The H2020 Grant Agreement includes optional provisions to protect the European interests such as Article 30.3 which requires beneficiaries to notify the EC before they transfer the results and gives the EC the right to object. This option is validated by default for the running and upcoming H2020 HPC projects. Any further measure to protect the European IPR is unlikely to be implemented before the end of H2020, as this would require an amendment of the H2020 rules for participation.

Could the shortcomings of the baseline scenario be effectively addressed by revising the existing instruments, i.e. amend the mandate of the HPC cPPP, extend the mission of PRACE, or evolve the H2020 rules and procedures? PRACE provides the most powerful supercomputers in Europe but PRACE does not aim to coordinate national programmes or investments in the procurement of HPC systems. PRACE was created in 2010 and a new agreement entered into force in 2016/2017. With the new agreement, there is a small financial contribution to PRACE of the members not hosting a Tier-0 machine. Eventually a new agreement could be negotiated to include the Union, represented by the EC, as a member; a coordination of the procurement; and a revision of the governance. However this is unlikely to happen because: (i) the hosting countries will continue to protect their national interests and not lose control over their Tier-0 supercomputers; and (ii) the PRACE machines are owned by the Tier-0 hosting countries and are designed and developed to primarily satisfy national research needs, in accordance with the relevant national strategies and programmes and do not necessarily cover wider European user needs. The mandate of the HPC cPPP is limited in time and is foreseen to be revised after 2020. The cPPP is playing a valuable role in defining and implementing the HPC strategic research agenda and leveraging private funding. One of its weaknesses is that its membership is open to non-EU actors. As a result the dominant, non-EU suppliers are today part of it, influencing the definition of the H2020 workprogrammes. This makes it more difficult for European market actors to develop European technology that finds its way into the subsequent procurement of machines. Finally, the use of different H2020 and CEF WPs for HPC and their challenging coordination can only be resolved by a revision of the H2020 Regulation, by integrating them into a single pillar. Such revision is not feasible any more. Even if it this was the case, it would resolve coordination of H2020 activities but not the coordination with CEF. Therefore, this option does not allow neither the pooling and efficient synchronisation of the EU investments (as they are split across multiple programmes and areas), nor the coordination of the EU activities funded by the EU budget with those of the MS. This kind of approach already proved to be insufficient in bringing about an EU-wide HPC ecosystem as this is also reflected in the 2017 EuroHPC Declaration by MS and further validated by the findings of the EuroHPC targeted consultation. Fragmentation of HPC programmes and efforts in Europe and the lack of coordination of activities was the main problem for the computing centres and the scientific users that responded to the public consultation. 5.2.Options considered for implementing EuroHPC but discarded at an early stage Option: EEIG The European Economic Interest Grouping (EEIG) is a cooperation form, similar to national consortia, but ruled mainly by European law. That means its core rules are common throughout the whole EU, and national rules are required only for a few issues (e.g. nullity, dissolution, some profiles of a grouping's administration). An EEIG is a type of legal entity designed to make it easier for companies in different countries to do business together or to form consortia to take part in EU programmes. A main usage of this supranational type of enterprise is to facilitate co-operation in European programmes especially for small and medium-sized enterprises (SMEs). The EEIG differs from other companies first and foremost in its purpose. It is supposed to help its members to develop or improve their own fields of activity. Thus, the EEIG can be used to develop certain common activities which would be too expensive for single members. Because of this auxiliary nature the activity of an EEIG has to be connected to the economic activity of its members. Not only SMEs, but basically every company or firm and all institutions governed by public or private law can be a member of an EEIG. The EEIG is eligible to participate in procurement procedures. However, the EEIG can only participate as a beneficiary. It cannot coordinate the procurements of the EC and the MS, nor can it implement a joint procurement. The procurement would still have to be implemented by the MS and/or the EC. As the EEIG has full legal capacity, its participation in EU R&I projects as partner or coordinator is possible. The EEIG is eligible to receive community funds. However, this option does not effectively address the problem of fostering the supply of European technology. As in the baseline scenario, the EEIG would be in competition with non-European applicants for the H2020 calls. Also the problem of ensuring continuity of the actions between subsequent calls remains as well as the problem that the EC has with the synchronisation of HPC investments under different H2020 and CEF WPs. Option: Galileo-type programme In a Galileo-like programme the EU has the sole ownership of the programme and delegates its implementation to a separate legal entity. In discussions with the EuroHPC signatory countries, they have clearly expressed their preference for a legal instrument where they would have co-ownership of the activities, in particular for the procurement and operation of the EuroHPC machines. The legal basic act setting up the Galileo-like programme would have to be followed by a delegated act to a legal entity for its implementation. The overall process is expected to take around 3 years. Such timing is incompatible with the EuroHPC plans as discussed above, making it impossible for the EuroHPC to reach the targets set for jointly procured pre-exascale systems by 2019-2020. It would also highly compromise all the longer term plans in developing and procuring exascale systems based on European technology by 2022/2023. Option: Intergovernmental organisations An intergovernmental organisation like the European Space Agency (ESA) is an agency established by a convention between its participating states, establishing a joint programme between Member States. The Member States signing EuroHPC have not proposed the creation of an intergovernmental organisation and the EC does not have the right in the Treaty to put forward such a legal form in a regulation. The EC can only participate in joint activities after negotiating a Cooperation Framework. As the EC would not be part of the governance the EC would have a limited influence on the definition of the calls for funding the joint activities, nor would the EC have a say on the acquisition and operation of the supercomputers. Finally, establishing such an organisation is unlikely to be operational by 2019, when the activities of the new legal entity have to start, as discussed above. In summary, this option would not lead to a legal entity that can jointly procure and operate HPC machines. The following table compares the baseline scenario and the discarded options against the functionalities which the new legal instrument should fulfil. As can be seen, none of them fulfils all the functionalities.





Baseline Scenario EEIG Galileo-type Intergovernmental Organisation Pool public and private funds 0 0 0 0 Execute joint procurements while operating under EU-law 0 0 OK 0 Implement a R&I programme OK maybe 0 OK Remain open to incorporate new MS/AC willing to join OK 0 0 OK Enable participation of the private sector OK OK OK 0 Safeguard the Union's interest through EC participation OK 0 OK 0 5.3.Option 1: ERIC The Community legal framework for a European Research Infrastructure Consortium (ERIC) is a specific legal form to facilitate the establishment and operation of research infrastructures with a clear European interest and the involvement of several European countries. The principal task of an ERIC is to establish and operate a research infrastructure on a non-economic basis. An ERIC can be used for establishing new research infrastructures or for operating existing research infrastructures which consider it useful to change their legal structure to become an ERIC. The ERIC can thus interconnect and federate regional, national and European HPC resources provided the research infrastructure meets the requirements set out in the ERIC regulation. The ERIC can provide the management of the coordination between the infrastructure and existing national computation resources and also, if agreed, regional computation resources. The ERIC framework has been developed primarily for new high-profile research infrastructures with a European dimension. Complementing national and inter-governmental schemes, the ERIC Regulation provides a common legal framework based on Article 187 TFEU. The ERIC has legal personality and full legal capacity recognised in all MS. Although an ERIC can be the beneficiary of H2020 grants, its current mandate does not allow it to get delegation to implement part of the H2020 programme. Therefore the ERIC would not be able to define a strategic R&I agenda and implement it, but would depend on the H2020 WPs defined and implemented by the EC. ERIC's basic internal structure is flexible, leaving the members to define the statutes, membership rights and obligations, as well as the bodies of the ERIC and their competences, while complying with the essential requirements provided for in the ERIC Regulation. The members of an ERIC can be MS, ACs, third countries other than the AC and intergovernmental organisations. The EU represented by the Commission may participate in an ERIC. However, the EU participation as a full member in the ERIC and the way it would make available its financial contribution are currently not foreseen and would have to be defined in some way. While the ERIC Regulation leaves open the possibility of the EC (as an international organisation) to become a member of an ERIC and to contribute through its membership to the ERIC, this has not been the case in any of the 17 ERICs that have been established until now. In practice, the participation of the EC in an ERIC in whatever form would create a situation of conflict of interest whenever the ERIC would participate in calls from EU programmes, as the EU would both be allocating budgets for the ERIC from EU programmes (notably H2020 and CEF) and would be co-responsible in a wise spending of such funds. The participation of industry is not foreseen in an ERIC. An ERIC as such is also closed to private funding, in turn restraining the participation and financial contribution by industry players to any future initiative. This is because the principal task of the ERIC is to establish and operate on a non-economic basis. In principle, ERICs are not designed to pool resources from the EU and MS but mainly from the MS themselves. Likewise, it would make it difficult for an ERIC to implement a joint EU/MS procurement. An ERIC is recognised by the country hosting its seat as an international body or organisation for the purposes of the directives on VAT and excise duties. It also qualifies as international organisation for the purpose of the directive on public procurement. An ERIC may therefore, under certain limits and conditions, benefit from exemptions from VAT and excise duties on its purchases in all EU MS and it may adopt procurement procedures respecting the principles of transparency, non-discrimination and competition but not subject to the directive on public procurement as implemented in national law. The advantage of the ERIC is that it can be setup with a rather lightweight and fast process. This would give confidence that by 2019 the ERIC would be operational and could implement, at least partially, the objectives of the HPC strategy. However, this would not solve some of the present shortcomings of the baseline scenario, like the difficult synchronisation of calls. To include in the mandate of the ERIC the possibility to implement H2020 calls on behalf of the EC this would require a revision of the ERIC regulation, which in turn is a lengthy process. Based on the above arguments, the table below summarises how the ERIC responds to the functional requirements the EuroHPC instrument should fulfil: ERIC Pool public and private funds ERICs are designed to pool resources mainly from the MS Execute joint procurements while operating under EU-law The EU participation as a full member in an ERIC is not foreseen, but under certain conditions the EC may participate. Joint EC/MS procurement is difficult Implement a R&I programme Current mandate does not allow ERIC to get delegation to implement part of the H2020 programme Remain open to incorporate new MS/AC willing to join OK Enable participation of the private sector Industry cannot become member of an ERIC Safeguard the Union's interest through EC participation The EU participation as a full member in an ERIC is not foreseen, but under certain conditions the EU may participate 5.4.Option 2: Joint Undertaking A Joint Undertaking (JU) is a Union Body established under Art 187 TFEU, which can be used for the indirect management of the EU budget. A JU is therefore an autonomous EU legal entity, with its own staff, budget, structure, rules and governance that can be tasked to implement actions under programmes such as H2020 and CEF. It can combine EU budget with other sources of funding (national, private, etc.), allowing the implementation of R&I and demonstration programmes in an integrated way. Public authorities at different levels from the MS (national, regional), the Union represented by the EC and other stakeholders like industry can become members of a JU. JUs have been used in FP7 and H2020 to establish the Joint Technology Initiatives (JTIs): Public-Private Partnerships (PPPs) that define and implement the Strategic Research Agendas (SRAs) of a limited number of areas where the scale and scope of the objectives is such that loose co-ordination through European Technology Platforms (ETPs) and support through the regular instruments of the Union's Research Framework Programme are not sufficient. According to their founding regulations JUs have their own governance structure but can also share some functions (e.g. internal audit) with other already existing JUs for efficiency reasons. Each JU includes a Governing Board, an Executive Director as well as other bodies, including advisory bodies, depending on its specific operational and governance needs.

JUs have a dedicated budget and staff and provide a framework for the public and, when appropriate, private players to work and take decisions together. They can organise calls for proposals and put in place implementation arrangements. They thus allow funds from different sources to be jointly managed and they are responsible for the related planning, monitoring and reporting activities. JUs can also carry out procurement procedures. Each JU has its own procurement and financial rules adopted by the Governing Board, based on the Union's model Financial Regulation. Established as a Union body, a JU can benefit from VAT and excise duties on its purchases in all EU MS and may adopt procurement procedures not subject to the Directive on public procurement as implemented in national law. JUs provide the legal, contractual and organisational framework to structure the joint commitments of public and private stakeholders and monitor the implementation against agreed Key Performance indicators (KPIs). In addition, they offer a firm governance structure and budgetary certainty to all stakeholders. JUs offer the possibility of joint activities between MS and the EU, allowing to keep e.g. MS/AC participation flexible according to country-specific priorities. Nevertheless, there is co-responsibility in the strategies and modes of implementation chosen. A report of the Estonian presidency reflected the very positive view that industry partners and researchers express on this kind of PPPs. The report indicates that such partnerships are a good instrument for strengthening the industrial base of Europe by connecting the European ecosystem to global companies and by encouraging good quality industry-led research which enables the EU to remain competitive and a leader in innovation and technology. There are several good reasons to establish a JU in the area of HPC: (i) the added value of the combining EU and national funding; (ii) the coordination and rationalisation of procurement and R&I investments at European level; (iii) the possibility to safeguard the Union's interests as the EU can be member of the JU, by exerting its decisional powers in the JU Governing Board; (iv) the possibility to combine innovation procurement instruments (PCP and PPI) to fund the development of innovative European technology and procure afterwards this technology; (v) the possibility