Data Spaces Symposium
HPC meets the demands ofAI and Data

Jeanette Nilsson RISE

Different perspective on AI, Data and HPC- connecting SME and AI community

 Juan Pelegrin, EC DG CNECT
 Roberta Turra, Cineca
 Andreas Wierse, SICOS
 Kristina Knaving, RISE

Jeanette Nilsson
AI/ HPC-Ecosystem expert
BDVA, Member of the Board of Directors of RIAG member
Sweden's representative in the EU's AI and Business Digitalization Working Group
with responsibility for the Partnership for AI, Data and Robotics (Adra)

AI Factories - Amendment to
the EuroHPC JU Regulation
Council Regulation (EU) 2021/1173

Juan Pelegrin
DG Communications Networks, Content and Technology
European Commission
12 March 2024

4

Large Artificial Intelligence (AI) models

 Large AI models (such as large language models) are a new wave of AI
models called generative AI adaptable to various domains and tasks.
 Generative AI models have immense potential to revolutionise multiple
sectors.
 Most large AI models (e.g. ChatGPT) are non-European
 Mastery of this technology is of strategic importance for Europe in linewith economic security.

5

Key AI applications
Robotics

Healthcare

Materials
and batteries

Biotechnologies

Manufacturing
and engineering

Public Sector

Climate change andadaptation

Cybersecurity

Mobility

Citiverse

Science6

Protein structure prediction

The greatest achievement of AI - and a paradigm shift in science

7

7

8

Towards large AI models:
European strengths and weaknesses
Strengths
70% of top-tier AI researchers in Europe work inacademia, 15% of all AI research publications from EU,
21.5% of all research citations in the field
Emerging AI ecosystems
Foundation models start-ups: Mistral, Aleph Alpha,Bloom, SiloAI & 125+ start-ups in general AI

World’s most advanced HPC infrastructure(EuroHPC) + the world’s largest database on
language models

Weaknesses

Skills, talents,

Vibrant start-ups
ecosystem
Advanced HPC
infrastructure and data
spacesCatching-up mode, full
reliance on non-EU actors
using black box AI models

EU reliant on non-EU companies toaccess general-purpose AI systems

AI ActAccess to VC financing

Limited access to venture capital above €10
million for start-ups + risk of take-overs

Excellent researchecosystem

Uncoordinated approach

Multiple but uncoordinated initiatives at EU/MS
level hampering effectiveness and scale-up

The world’s first comprehensive regulatory approach to AI
Excellent European research ecosystem;
multiple AI research initiatives across MS with EU second
largest in research publications (2022)

9

Context•

Generative AI and AI foundation models are advancing at unprecedented paceand are set to play a pivotal role in shaping the future of technology and society.

•

AI is a key policy area of the EU digital strategy. AI in combination with HPC cancontribute to a more innovative, efficient, sustainable and competitive economy,
while also improving safety, education and healthcare for citizens.
• In her 2023 State of the Union address, President von der Leyen
announced that the supercomputing resources of theEuroHPC JU will be made available to European AI startups
to train their large-scale models, contributing to the EU’s aim ofleading global advances in AI and of achieving responsible andethical innovation.

10

Unlocking the potential of AI
AI

providers(models,algorithms,
…)

Data

AI (GPAI, FM,
LLM, …)
leadership

HPC

Algorithms
Talent &Competences

11

11

EuroHPC for large AI models (estimation)(18 petaflops peak
performance)(550 petaflops peak
performance)(323 petaflops peak
performance)

Leonardo – IT*

Jupiter-DE
Approx. figures

Falcon40 bn parameters

50 days

4 days3 days

1 day

LLaMA (META)
65 bn parameters

80 days

7 days

5 days

1.5 days

GPT-4 (OpenAI)
Hundreds of bn
parameters

6 years

6 months

4 months

6 weeks

HPC system /AI model

30 BN

70 BN

200 BN

Source: Internal analysis

Meluxina - LU

LUMI - FI

How to capitalise on EuroHPC strengths to develop a highly
competitive and innovative AI start-up and research ecosystem
in Europe?
• Procurement of supercomputers.
• Upgrading of existing EuroHPC supercomputers.
• Access to supercomputers.
BUT:− Current EuroHPC infrastructure and services not optimized for AI.
− Inflexible conditions for upgrading.
− Not considering specific needs of AI user communities.
− No connection with AI ecosystem.
 A targeted amendment to the EuroHPC Regulation is necessary.

14

AI Factories
Talent &Competences

Data
centreOne Stop-Shop

Access &AI-HPC
services

Data
DataSpacesSpacesThe EU’s AIoriented
supercomputersAI
Facilities
supercomputer-friendly
programming
facilities

15

Amendments to EuroHPC Regulation(EU) 2021/1173
AI Factory - Activities
1. Acquisition & operation of AI dedicated supercomputers (co-located with data centre)
2. Upgrading with AI existing EuroHPC
3. Provide access to SMEs and start-ups (incl. widening usage)
4. AI supercomputing service centre (algorithms, training- testing- evaluation- validation of AI models,development of large-scale AI applications, …)
5. Supercomputer-friendly programming facilities (parallelization, usage optimization, …)
6. Attracting & pooling talent
7. Interacting with AI-ecosystem at large & other AI initiatives

Hosting AI supercomputers conditional of establishing AI Factory16• Hosting Entity

AI Factory• AI-Factory related Hosting Entity eligibility criteria / Hosting Entities => One-Stop-Shop• Hosting Agreement• Specific hosting conditions for AI-dedicated supercomputers / AI-upgrades• Acquisition and ownership of Artificial Intelligence-dedicated supercomputers• Apply for hosting anytime => no limitations as for high-end/mid-range• Upgrading of Supercomputers• Deleting € limit/ Widening scope to request upgrades/ Apply for upgrade anytime => no limitations to apply
within 3 years after selection as hosting entity• EU financial contributions (OPEX/CAPEX) percentage as for original supercomputer to be upgraded• Use of EuroHPC Supercomputers• Governing Board defines special access conditions for AI supercomputers / upgrades• Allocation of Union Access time to EuroHPC supercomputers• EU share of access time proportional to EU contribution

THANKS17

HPC meets thedemands of AI and
Data

Leonardo: Bridging AI
Needs, Access, and HPC
Evolution
12/03/2024 15:30–16:30 (CET)
Roberta Turra, CINECAThe impact of AI on HPC can beseen from different perspectives• Tasks / Requirements• Users• Access to resources• Hardware• Software• Projects / Apps
• ServicesTasks / requirements
What’s new in HPC (impact
Traditional HPC

of AI)• Run physical (deterministic)• Data driven ML algorithms

models that generate data

• Applications: SIMULATIONS(Computational Fluido
Dynamics, Molecular
Dynamics, … )(probabilistic) that generate themodel• New workload need optimization of
HPC systems for AI tasks• New applications on texts andimages• With Deep Learning computational
power becomes one key element(data – algorithms – compute)LLM’s evolution

The number of model
parameters—whichcharacterize the models' sizeand some of its power—roseby about 25,000 times
between 2017 and 2024.

UsersTraditional HPC

What’s new in HPC(impact of AI)• Domain: Natural• Domain: Social

sciencessciences (in the
beginning)• Main typologies:• Researchers (scientific • No single community
communities)• Big companies (e.g.
pharmaceutical, Oiland Gas, …)• New typology: startups

CINECA HPC Annual Report 2022/23

Access to resources
What’s new in HPC (impact
Traditional HPC

of AI)• Scientific merit (with• Specific calls

limitations on the number ofcore/hours)• Contractual / research
agreements• Public funding (e.g. EU
projects)• EuroHPC• AI-BOOST• ALT-EDIC• EuroCC2, FF5, EDIHs, … (in Italy the
National Competence Center for HPC,ICSC)

• Specific agreements(in Cineca: FAIR, FBK, AL, IBM,
iGenius)Hardware
Traditional HPC

What’s new in HPC (impact
of AI)• CPUs• GPUs• Double precision• Bandwidth• (fast)Storage

AI and machine learning workloads have driven the development and adoption of
specialized hardware such as GPUs (Graphics Processing Units), TPUs (Tensor Processing
Units), and FPGAs (Field-Programmable Gate Arrays). These accelerators are designed
to perform parallel operations at high speeds, essential for the massive matrix and vector
computations common in AI. This shift has led to a rethinking of HPC infrastructure toaccommodate these accelerators, leading to more heterogeneous computing
environments.

Tier0 Leonardo + Lisa AI
upgradeLeonardo
Data Centric Module

Lisa
AI Module
PODS SMP Cluster

1536 compute nodes
2x Intel SPR CPUs,
512 GB DDR5, NVM

Low Latency Interconnect 400Gb/s

DC Casalecchio

Storage Fast Tier
5.4 PB, 1.4 TB/s

Storage data-lake Tier
100 PB

Storage High performance
Tier
106 PB, 620 GB/s
Ethernet Interconnect 400Gb/s

Leonardo

Ethernet

Lisa

InfiniBand

Leonardo
Booster Module3456 compute nodes
4x NVIDIA A100-64 GPUs

Low Latency Interconnect 200Gb/s

Front-end
partition

Login nodes
Visualiz. nodes

Gateway
2.5 Tb/s

Facility distribution and routing

Internet and Geant

HPC infrastructure
Quantum
computing

HPC for AI

Multi protocol
multi Petabyte
data lake

Cloud service evolution

Cloud• Flexible use of resources (dynamic resource allocation, allowing users toscale resources up or down on-demand)• Interactive Computing Services (enable to provide front-end services forthe users, web access, real-time data processing, graphical interfaces for
inference …)• Environment personalization (operating system and software stack canbe selected and installed by the user)• Security (GDPR compliant)Software
Traditional HPC

What’s new in HPC (impact
of AI)• Specialized software libraries,• Pytorch, tensorflow

compilers, and tools optimized
for performance on specifichardware architectures (e.g.
Quantum espresso)• Repositories• New libraries• New modules (e.g. CinecaAI)• Data Lake

Projects / Apps

Traditional HPC•Particle Physics•Condensed Matter Physics•Astrophysics and Plasma Physics•Computational Chemistry•Computational Engineering•Computational Fluid Dynamics•Earth and Climate Science•Life Science – Computational biology•Life Science – Bioinformatics

What’s new in HPC (impact
of AI)• A new scientific area in projectclassification: AI & Machine learning• Scientific and innovation(commercial) projects that either
develop new AI algorithms or use
AI in their application domainProjects / Apps
National research projects (ISCRA C)
running on Cineca infrastructure in
2023: focus
Project# of abstracts

AI & MLBig Data/Bioinformatics
COVID-19
HPC
Quantum Computing
Total

101
39
7
581
18
746

Projects / Apps
Word cloud in the ISCRA C project titles

Climate and Weather
projects• CYBELE (www.cybele-project.eu/)
Day-ahead hail prediction to prevent losses in agriculture• DAMAS (https://euhubs4data.eu/experiments/damas/)
Data-driven Model for the Analysis of Sea-state• Climate Byte (https://euhubs4data.eu/experiments/climaterisk-assessment-for-agriculture-insurance-using-big-dataand-artificial-intelligence/)
Climate risk assessment for agriculture insurance• ILLUMIA(https://www.ifabfoundation.org/ifab-activities/projects/aigeneral-circulation-model/)AI General Circulation Model• OptimESM (https://optimesm-he.eu/ )Optimal High Resolution Earth System Models for Exploring
Future Climate Change• Weather4Energy(https://www.supercomputing-icsc.it/en/spoke-4-earthclimate-en/)Earth Observation
projects• Arboria (https://euhubs4data.eu/experiments/arboria/)
An open-data based urban forest solution improved by AI
optimization• Digital Twin of Bologna city
• Mobility Square(https://euhubs4data.eu/experiments/sustainablemobility-recommendation-engine-for-tourism/)
Sustainable mobility recommendation engine for tourism

Other relevant ongoing projects
Cataloguing and meta-dating of artistic heritage
Support the Ministry of Culture in the cataloguing and metadating of artistic heritage through the use of image analysis
techniques (object detection, classification, captioning)

Legislative support using artificial intelligence and
LLMs
Design and implementation of a semi-automatic laws impact
evaluation system that can generate new high quality text laws
and improve impact

Retrieval Augmented Generation for Systematic
Literature Review

Natural Language Processing for healthy life expectancy
estimation
Data Centre Digital Twin for Sustainable ExascaleComputing

Design massive Data Center graph models capturing the spatiotemporal dependencies
between computation, nodes, and cooling equipment and conduct analytics to predict the
impact of the spatial power distribution on cooling efficiency and cost (https://graph-massivizer.eu/project/data-center-digital-twin/)Services
Traditional HPC

What’s new in HPC (impact
of AI)• User support• Data catalogue• Specialistic support• Free inference for widely• Training• PoC

adopted open-source models• Fine-tuning•…

Challenges• Ethics• Sustainability

What’s next
• The transformation is goingon• We learn by doing, by actively
participating to AI projects
and by providing user support
in the most computationallychallenging tasks (e.g. LLM)

Thank you!

Bridging HPC Access for SMEs
EuroCC Initiatives and the German
Ecosystem
Dr. Andreas Wierse

Managing Director SICOS BW GmbH

1985
Lothar Späth buys a Cray 2 ...

1995
Foundation of hww and HLRS

2011Foundation SICOS BW GmbH

Focus: SMEs... and creates 10 new
positionsfor SME consultants

The SICOS BW Tasks
Facilitate SME-Access to HPC Systems
and Large Scale Data Facilities
Support SMEs in finding
Competence Partners
Support the Centers in the Optimisationof their Offering towards Industry

SME

From Concepts to Implementation• EuroCC kicked-off as Research & Innovation Action
in September 2020 (End of Phase 1: 31.12.2022)- Phase 2 started 1st of January 2023 (duration 3 years)• 32 nations
28 Beneficiaries, 60 Affiliated Entities, 11 Associated Partners• Germany, Bulgaria, Austria, Croatia, Cyprus, Czech Republic,
Denmark, Estonia, Finland, Greece, Hungary, Ireland, Italy,
Lithuania, Latvia, Poland, Portugal, Romania, Serbia, Slovenia,
Spain, Sweden, France, Netherlands, Belgium, Luxembourg,
Slovakia, Norway, Turkey, Republic of North Macedonia,
Iceland, Montenegro• Funding: 50% EuroHPC JU, ~50% from the respective
states• https://www.eurocc-access.eu/• LinkedIn: EuroCC, Twitter: @EuroCC_projectWhat is the Background of EuroCC?
European High-Performance Computing Joint Undertaking ('granting
authority‘),under the powers delegated by the European Commission ('European
Commission‘)
The Topics:• HPC – High-Performance Computing• HPDA – High-Performance Data Analytics• AI – Artificial Intelligence
The mission of EuroCC 2 is to continue the establishment of a network of
National Centres of Competence (NCC) in the most efficient way, while
continuing to address the differences in the maturity of HPC deployment inEurope, for which improvement has already been noted.

What are National CompetenceCenters?
National Competence Centers (NCCs) are organizations or institutions
that are designated as experts in a particular field or technology bya national government. The purpose of NCCs is to coordinate andenhance the development of national expertise and capabilities
in a specific area.
They typically play a key role in promoting research,development, and innovation, and may also be involved inproviding training, education, and other services to support thegrowth of a particular industry or field.
NCCs may be established in a variety of fields, such as advanced
manufacturing, biotechnology, renewable energy, or information
technology.
They may work with universities, research institutes, andprivate sector companies to support the development of newtechnologies, products, and services. NCCs are often funded by thenational government or through partnerships with industry andacademia, and may have a variety of goals, including improving thecompetitiveness of national industries, promoting economic

Contact
SICOS BW GmbH
Nobelstraße 1970569 Stuttgart
www.sicos-bw.dewierse@sicos-bw.de
0711-342033-0

Safeguarding
the Information
Ecosystems
Kristina Knaving
Senior Researcher, RISE

Misinformation and desinformation are poisoning our
information ecosystems•

Individuals as well as malicious actors•

Internet, intranets, research•

Human information gathering at risk, also LLM summarising of content•

Model collapse and feedback loops

Submissions to Clarkesworld

Bias

Bias

Bias exists in all data and all humans, andcannot be entirely removed….
…but model bias likely to continue to be lessdiverse than human bias.
We often focus on easily perceived biases”Microbias” can be a problem over many
interactions

Generalisation and convergence –”The AI SAMENESS”
LLMs turn statistical majorities into absolutes

Semi-synthetic data•

AI-powered tools makes ”human-made” content
into semi-synthetic–•

Often without users’ knowledge

Number of tools with AI support grow rapidly- AI
filters, adjusts, suggests, generates

Safeguarding the information
ecosystems
Suggested solutions•

AI detectors– Risk for false positives and negatives•

Mandatory labelling , watermarking and shibboleths– Only works for regulated models and model use•

Known sender– Privacy issues•

Human attention is needed, even though expensive•

Trusted repositories, verified data sets, curated content•

Educating users to facilitate human-in-the-loop•

Technical soluutions, such as expand modalities

Trusted data repositories• Curated content / verified sender• For information gathering, but also for training models• Who decides what is trustworthy and less biased?
• Who owns the data? Will usable data be locked into organisations?
• Who will be able to train on especially valuable data, such as research
articles?

Panel and Summary