November 2017

EPCC IN 2017: HPC AND DATA
Professor Mark Parsons
EPCC Director
Associate Dean for e-Research1November 2017

EPCC in 2017
• One of Europe’s top Supercomputing centres• 27 years old – ~100 staff• Fully self-sustaining - £13m turnover in FY16/17• UK National HPC Service provider• Wide range of work from HPC to Data Analytics

and Cloud• Two MSc programmes – “HPC” and “HPC with

Data Science”• Known worldwide for our industry collaboration

programmes in HPC and Data Analytics• Well over a thousand companies since 1990

2

November 2017

3People are our most important asset … not computers

Almost all ofthese people
have worked for
EPCC over thepast 25 years

30th November 2017

Bayes Centre for Data Technologies• New £43 million building in centralEdinburgh• EPCC has taken whole floor• Room for 130 people• Next to School of Informatics• Brings together many data related

activities under one roof
• First time EPCC has moved in 27years …

5November 2017

EPCC structure in 2017
••
•
••
•

Senior Management
System Developer TeamACF Team
Administration
Applications GroupCommercial GroupNovember 2017

Principal services• ACF houses variety of leadingedge systems and infrastructures• UK national services• ARCHER 118,080 cores (Cray XC30)• DiRAC 98,304 cores (IBM BlueGene/Q)• Funded by EPSRC and NERC
• UK Research Data Facility (25Pb Disk / 50Pb Tape)
Tappe)• Service opened in 2013
• Cirrus – Tier 2 HPC and• Industry
machine5,053 users
since opening• 3,494users in past 12 months• Scottish National Data Safe
Haven

• Local services• ULTRA – SGI UV2000• ECDF – Compute and data store

clusters for University researchers

6

November 2017

Newest system – Cirrus• New SGI ICE XA system• Now called an HPE 8600

• Bought for• EPCC industry activities• Edinburgh Genomics – Scotland’s“Whole Human Genome Factory”• Expanded in March 2017 to 13,000+

cores as part to become EPSRC
National Tier 2 HPC service• Includes new Tier 2 data store

79th June 2017Managing Personal Health Data for Research

Systems grow very quickly – Schematic layout April 2016

110 TB LFS

800 TB LFS

89th June 2017Managing Personal Health Data for Research

9Schematic layout September 2016

800 TB LFS

9th June 2017Managing Personal Health Data for Research

10

Schematic layout March 2017 - computeFrom
5,184 coresTo
13,248 cores9th June 2017Managing Personal Health Data for Research

11

Schematic layout March 2017 - storage

1.9PB WOS

November 2017

Data hosting – building capacity and skills• Since 2015 have hosted the National

Data Safe Haven for Scotland• Multiple datasets• Enables safe research with unconsented

personal data e.g. health records• Controlled by Scottish Government’s “publicbenefit and privacy” policy

1230th October 2017

Meet the Teams

National Safe Haven – pseudo-anonymisation process
All Data Controllersfollow the sameprocess as for Data
Controller 2

Data Controller 1

1U2

Data and index numbers sent
securely to Linkage Agent;
participants have different index
number in each dataset

2Indexing Service
returns unique index
number for eachparticipant

3

U1: Approved Researcher
U2: Data Provider
U3: Tech Support
U4: Indexer
U5: Research Coordinator
U7: Vendor Support
U8: Linkage TeamNo accesswithout
DataController
permission

Indexing Services at National
Records Scotland and NHS
National Services Scotland

4National Safe Haven
7

U3
U7
StatisticalDisclosureControl6Linkage Agent

U8StudyStudyStudyworkspace
workspaceworkspace

Nothing can leave the secure
analytic environment before it
is checked for disclosure risks

Trusted 3rd Party

U4

Internet/JANET

Secure Data
Archive forcompletedstudies

Data Controller securely sends identifying data tothe indexing service (e.g. Names, Post Codes, DoBs)

Data Controller 2Indexing Service securely
sends the master indexing
key to the Linkage Agent

Secure thinclient access

5

Secure Access
PointU1
U513

November 2017

Edinburgh Region City Deal – a key part of our future• In 2016 EPCC helped develop a“Science and Innovation Audit”• Identified strengths in our region

for Data Driven Innovation• City Deals are funding from UKand Scottish Governments• Aim is to stimulate economicgrowth• £1.1 billion Edinburgh Region City

Deal announced in summer 201714

November 201715

Aims of City Deal
• Capitalise on our expertise in Data Driven Innovation• Make Edinburgh City Region the “Data Capital of Europe”• Create a trusted public-private-third sector partnership• Unlock economic opportunities worth £5 billion+• Train 100,000 people in data technologies• Develop an underpinning infrastructure – the World Class DataInfrastructure (WCDI)

November 2017

City Deal outline

16

November 201717

World Class Data Infrastructure• City Deal includes capital investment in WCDI
• New high resiliency data centre room, computers, storage,

networking and software• Will support work with complex, high volume, real-time datasetsfrom across City Region and beyond• Already demand - FinTech community, GCHQ, DSTL, NRS, HSBC…
• All 10 sectors targeted through City Deal will need access• Including Local Authorities, local companies etc• Building a data hub, creating new applications … and companies

Fortissimo’s Goal & Ambition• Goal: provide SMEs with easy and cost-effective access toadvanced simulation services through a Cloud infrastructure
consisting of HPC resources, software applications,expertise, and tools• Ambition: become THE portalof choice for HPC and HPDA
expertise and service provision,delivered by Europe’s major
HPC technology providers
November 201718

Fortissimo projects in numbers• Fortissimo - €22m FP7 project – ended 12/2016– 122 partners– 53 ‘experiments’ in three tranches delivering real impact– Focus on HPC enabled modelling and simulation for manufacturing SMEs andMid Caps

• Fortissimo 2 - €11m H2020 project – ends 10/2018– 93 partners– 39 ‘experiments’ currently running– Fortissimo focus plus High Performance Data Analytics• Lots of effort to help SMEs take part– Particularly with respect to IPR management and financeNovember 201719

Similar model for both projects• Small set of core partners– Almost identical for both projects• Initial set of ‘experiments’• Two Open Calls for experiments– At Month 6 and Month 12• Experiments last 18 months andinvolve 3-5 partners and funding
up to €250,000
November 201720

Cloud-based simulation of continuous casting• CFD modelling liquid steel pouringfrom ladle to tundish• Aim to minimise slag transfer• Fast return on investment• Medium sized steel plant produces 1mtons steel per year
• Operating costs of €300 million• Estimated €3 million annual saving• Now being exploited by Ergolines
November 201721

Cloud-based CFD simulation
for hypercars• Koenigsegg are EU Hypercar
manufacturer … and an SME• In-house CFD too expensive– Cloud is compelling option• Impressive results– 250% increase in downforce with only 15% increase in drag at 250kph• 30% saving in design costs plus 50% reduction in wind tunnel andphysical testing• Development savings of €90K per year PLUS 30% decrease in time tomarket• €4m benefit to company over 5 years
November 201722