Addressing Big Data Issues inScientific Data Infrastructure

Yuri Demchenko,
SNE Group, University of Amsterdam
21 May 2013, BDDAC2013 Workshop
CTS2013 San DiegoOutline• Background to Big Data research at SNE/UvA• Big Data definition– 5 V’s of Big Data: Volume, Velocity, Variety, Value, Veracity• Use case: High Volume Low Value (HVLV) data for financial market feeds– Will Big Data term and definition change/evolve?

• Big Data technologies landscape• Big Data and Data Intensive e-Science– Scientific Data Lifecycle Management• Scientific Data Infrastructure (SDI) for Big Data–••

Cloud and Intercloud based platform for SDI

Infrastructure research on SDI for Big Data
Discussion

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

Slide_2

Background to Research on Big Data
• System and Network Engineering Group at the
University of Amsterdam– Optical and high performance networks– E-Science and Collaborative applications– Computer Grids and Cloud Computing– Security, access control, trust management• Future Scientific Data Infrastructure– Fusion of industry Big Data and Data Intensive Science• Big Data requires infrastructure– High performance computing and network– Access infrastructure and collaborative environment– Clouds (and Grids) is not a full answer

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure3

Big Data and Data Intensive Science &Technology - The next technology focus
Scientific and Research Data – e-Science• Big Data is/has becoming the next buzz word
– Not much academic research and papers – Dive into blogs and tweets• Based on the e-Science concept and entire information and artifacts
digitising– Requires also new information and semantic models for information
structuring and presentation– Requires new research methods using large data sets and data mining• Methods to evolve and results to be improved• Changes the way how the modern research is done (in e-Science)– Secondary research, data re-focusing, linking data and publications• Big Data require infrastructure to support both distributed data(collection, storage, processing) and metadata/discovery services– Demand for trusted/trustworthy infrastructure– Clouds provide just right technology for (data supporting) infrastructure
virtualisation
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

4Big Data Challenges and Initiatives• A Vision for Global Research Data Infrastructure(http://www.grdi2020.eu/)– Final Roadmap Report published• Peta and Exa scale problems: Storage, Computing, Transfer/Network– International Exascale Software Project (http://www.exascale.org/)• International Initiative “Research Data Alliance (RDA)”
http://www.rd-alliance.org/ launched 2-3 Oct 2012, Washington– To accelerate international data-driven innovation and discovery by facilitating
research data sharing and exchange, use and re-use, standards
harmonization, and discoverability– Consolidated previous initiatives• Data Web Forum (DWF) initiated by NSF• DAITF – Data Access and Interoperability Task Force initiated by EUDAT
project– First meeting – RDA Official Launch, Gothenburg 18-20 March 2013
– Next meeting 15-17 September 2013, Washington

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

5Big Data Definition (1)•IDC definition (conservative and strict approach) of Big Data:"A new generation of technologies and architectures designed to economically extract
value from very large volumes of a wide variety of data by enabling high-velocity
capture, discovery, and/or analysis"•

Big Data: a massive volume of both structured and unstructured data that is so largethat it's difficult to process using traditional database and software techniques.
–•“Data that exceeds the processing capacity of conventional database systems.
The data is too big, moves too fast, or doesn’t fit the structures of your databasearchitectures. To gain value from this data, you must choose an alternative way toprocess it.”–•

From “The Big Data Long Tail” blog post by Jason Bloomberg (Jan 17, 2013).
http://www.devx.com/blog/the-big-data-long-tail.html

Ed Dumbill, program chair for the O’Reilly Strata Conference

Termed as the Fourth Paradigm *)“The techniques and technologies for such data-intensive science are so different that
it is worth distinguishing data-intensive science from computational science as a new,fourth paradigm for scientific exploration.” (Jim Gray, computer scientist)*) The Fourth Paradigm: Data-Intensive Scientific Discovery.
Edited by Tony Hey, Stewart Tansley, and Kristin Tolle. Microsoft, 2009.
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

6

Big Data Definition (2) - Motivation• Why Big Data based decision making support is so important?
– Read book “Thinking. Fast and Slow” (2011) by Nobel Memorial Prize winner
in Economics Daniel Kahneman• Our thinking is systematically mistaken when it comes to (subconscious) statistical
data assessment• Dutch translation “Ons feilbare denken” (“Our mistaken thinking”)• Blog article by Mike Gualtieri from Forrester:“Firms increasingly realize that [big data] must use predictive anddescriptive analytics to find nonobvious information to discover value inthe data. Advanced analytics uses advanced statistical, data mining andmachine learning algorithms to dig deeper to find patterns that you can’t
see using traditional BI (Business Intelligence) tools, simple queries, orrules.”

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure7

5 V’s of Big Data

Volume••••Terabytes
Records/ArchTransactions
Tables, Files

Variety••••

Velocity
•
•
•
•

Batch
Real/near-time
Processes
Streams

5 Vs ofBig Data

Structured
Unstructured
Multi-factor
Probabilistic
•
•
•
•
•

Value
•
•
•
•

Statistical
EventsCorrelations
Hypothetical

Trustworthiness
Authenticity
Origin, Reputation
Availability
Accountability

Commonly accepted
3V’s of Big Data

Veracity

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure8Volume, Velocity, Variety – Examples e-Science• Volume – Terabyte records, transactions, tables, files.
– LHC – 5 PB a month (now is under re-construction)– LOHAR – 5 PB every hour, requires processing asap to discard noninformative data
• Other astronomy research– Genome research– Earth, climate and weather data

• Velocity – batch, near-time, real-time, streams.
– LNC ATLAS detector collect about xTB data from X sensors during
the collision time about 1? ms– What other research require highspeed data

• Variety – structures, unstructured, semi-structured, and all
the above in a mix– Biological and medical, facial research– Human, psychology and behavior research– History, archeology and artifacts
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure9Volume, Velocity, Variety – Examples Industry• Volume – Terabyte records, transactions, tables, files.
– A Boeing Jet engine produce out 10TB of operational data for every 30
minutes they run– Hence a 4-engine Jumbo jet can create 640TB on one Atlantic crossing.
Multiply that to 25,000 flights flown each day and we get the picture• Velocity – batch, near-time, real-time, streams.
– Today’s on-line ad serving requires 40ms to respond with a decision– Financial services (i.e., stock quotes feed) need near 1ms to calculate
customer scoring probabilities– Stream data, such as movies, need to travel at high speed for proper
rendering• Variety – structures, unstructured, semi-structured, and all theabove in a mix– WalMart processes 1M customer transactions per hour and feeds
information to a database estimated at 2.5PB (petabytes)– There are old and new data sources like RFID, sensors, mobile
payments, in-vehicle tracking, etc.

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

10

Big Data – Veracity• Trustworthiness and Reputation -> Integrity• Origin, Authenticity and Identification– Identification both Data and Source– Source: system/domain and author– Data linkage (for complex hierarchical data, data provenance)• Availability– Timeliness– Mobility (mobile/remote access; from other domain – roaming;federation)• Accountability– A kind of pro-active measure to ensure data veracity

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

11

Big Data Challenges - Technological• How to scale up and down (scale or shrink)?
– Primarily database issues– SQL scales easy up but not easy scales down if demand decreases• NoSQL (Not only SQL) can partly address this issue• SQL has complex syntax, strong schema typing, performance– NoSQL is more flexible to adopt to new biz processes• Primarily but not just key-value or document-based• Data structures and data models– To respond to specific use cases and operations over data

• Data mining/data intelligence algorithms– To handle/discover new data structures and multi-type data relations– Human/behavioral/social targeted data analysis (means fuzzy/biased)• Infrastructure support for storing, moving data, on-demand
processing– Is Cloud Computing a right technology? Any alternative?
– High speed network infrastructure, on-demand provisioning• Big Data security, trustworthiness and data centric security
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

12

Big Data Challenges – Socio-technological• Extending big data outreach/perimeter– Technology will boom if there is sufficient customer and user base
• Currently majority of Big Data consumers are big companies– Although we are contributing with feeding our activity/usage log data

– Move big data from big companies to user and homes• Smart homes, sensors and devices– Without sensors and devices human can not create or use big data
– Smart visualisation can solve a problem of using/acting on big data

• Lowering entry level to use Big Data
– You should not be a data expert to use Big Data
– Needs for scalable configurable tools• Big Data and Privacy issues– Digital footprint and re-identification

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure13http://mattturck.com/2012/10/15/a-chart-of-the-big-data-ecosystem-take-2/

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

14

Big Data Infrastructure
Components
•

•
•
•
•
•
•
•
•
•

Cloud base infrastructure services fordata centric applications (storage,
compute, infrastructure/VM
management)
Cluster services
Hadoop related services and tools
Specialised data analytics tools (logs,
events, data mining, etc.)
Databases/Servers SQL, NoSQLMPP (Massively Parallel Processing)databases
Big Data Management
Registries, Indexing/search, semantics,
namespaces
Security infrastructure (access control,
policy enforcement, confidentiality,
trust, availability, privacy)
Collaborative environment (groups
management)

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure15

Will Big Data term sustain? – Other names• Big Analytics, Big Data Analytics– To avoid the term itself becomes a “fetish”• Data Analytics, Intelligent Analytics– Missed infrastructure component• Big Data vs Data Intensive Science– e-Science is based on and involves wide cooperation between
researchers• New concepts related to Big Data
– Disposable Data – in contrary to data supposed to be stored– Non-deterministic nature of the scientific study objects• From natural science to economics and social science

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

16

Big Data and Data Intensive Science

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure17e-Science Features
•

•

•
•
•

•

Automation of all e-Science processes including data collection, storing,classification, indexing and other components of the general data curation andprovenanceTransformation of all processes, events and products into digital form bymeans of multi-dimensional multi-faceted measurements, monitoring andcontrol; digitising existing artifacts and other content
Possibility to re-use the initial and published research data with possible data
re-purposing for secondary research
Global data availability and access over the network for cooperative group of
researchers, including wide public access to scientific data
Existence of necessary infrastructure components and management tools thatallows fast infrastructures and services composition, adaptation andprovisioning on demand for specific research projects and tasks
Advanced security and access control technologies that ensure secure
operation of the complex research infrastructures and scientific instruments andallow creating trusted secure environment for cooperating groups andindividual researchers.

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure18

Scientific Data Types
EC Open Access Initiative
Requires data linking at alllevels and stages

Publicationsand Linked
Data

Published Data
Structured Data
Raw Data

21 May 2013, BDDAC2013• Raw data collected from observationand from experiment (according to an
initial research model)• Structured data and datasets that went
through data filtering and processing(supporting some particular formal
model)• Published data that supports one or
another scientific hypothesis, research
result or statement• Data linked to publications to support
the wide research consolidation,
integration, and openness.

Big Data & Sci Data Infrastructure19

Traditional Data Lifecycle Model
Data collection
Data processing
Publishing research results
Discussion
Data and publications archiving

Lack of initial and intermediate data
preservation; and linking data to
publications

DataarchivingRaw Data
Experimental
Data

Project/
Experiment
Planning

Data collection

Structured
Scientific
Data

Data
Integration andprocessingDataarchiving

Publishingresearch
results

End of project:
Data discard orArchive

Open
PublicUse

Researcher

21 May 2013, BDDAC2013

DB

•
•
•
•
•

Big Data & Sci Data Infrastructure

20

Scientific Data Lifecycle Management (SDLM)
Model
Data Lifecycle Model in e-Science

User
Researcher

Data discovery

Data Curation(including retirement and clean up)
Data
recycling

Raw Data
Experimental
Data

Project/
Experiment
Planning

Data collection
andfiltering

Structured
Scientific
Data

Data analysis

DB

Data archiving

Data
Re-purpose

Data linkage
to papers

Data sharing/
Data publishing

Data Re-purpose

Data Linkage Issues•
Persistent Identifiers (PID)•
ORCID (Open Researcher andContributor ID)•
Lined Data

Data Clean up and Retirement•
Ownership and authority•
Data Detainment

Big Data & Sci Data Infrastructure

End of project

Open
PublicUse

Data Links

21 May 2013, BDDAC2013

Data
archiving

Metadata &
Mngnt

21

General requirements to SDI for emerging Big
Data Science
•
•
•
•
•
•

Support for long running experiments and large data volumes generated
at high speed
Multi-tier inter-linked data distribution and replication
On-demand infrastructure provisioning to support data sets andscientific workflows, mobility of data-centric scientific applications
Support of virtual scientists communities, addressing dynamic usergroups creation and management, federated identity management
Support for the whole data lifecycle including metadata and data source
linkage
Trusted environment for data storage and processing–••

Research need to trust SDI to put all their data on it

Support for data integrity, confidentiality, accountability
Policy binding to data to protect privacy, confidentiality and IPR

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

22

Defining Architecture framework for SDI and Security• Scientific Data Lifecycle Management (SDLM) model• e-SDI multi-layer architecture model• RORA model to define relationship between resources and actors– RORA (Resource-Ownership-Role-Actor) model defines relationship between
resources, owners, managers, users– Initially defined for telecom domain– New actors in SDI (and Big Data Infrastructure)• Subject of data (e.g. patient, or scientific object/paper)• Data Manager (doctor, seller)• Security and Access Control and Accounting Infrastructure (ACAI)– Trust management infrastructure– Authentication, Authorisation, Accounting• Supported by logging service– Extended to support data access control and operations on data

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

23

SDI Architecture Model

ScientificDatasetApplicScientific
ApplicScientific
ApplicScientific

Policy and distributed Collaborative
Groups Support

Layer B5
Access andFederation Layer

Shared Scientific Platform and Instruments(specific for scientific area, also Grid based)

Layer B4
Scientific Platform
and Instruments

Technologies andsolutions
Scientific specialist
applications
Library resources

Optical Network
Infrastructure
Federated Identity
Management:
eduGAIN, REFEDS,
VOMS, InCommon
PRACE/DEISA

Cloud/Grid Infrastructure
Virtualisation and Management
Middleware
ComputeResources

Sensors andDevices

Network infrastructure

21 May 2013, BDDAC2013

Layer B6
Scientific
Applications

User/Scientific Applications Layer

User
portals

Metadata and Lifecycle Management

Security and AAI

Operation Support and Management Service (OSMS)

LayersLayer B3
Infrastructure
Virtualisation

Middleware
security

Layer B2
Datacenter andComputing Facility

StorageResources

Layer B1
Network Infrastructure

Big Data & Sci Data Infrastructure

Grid/Cloud

CloudsAutobahn, eduroam

24

SDI Architecture Layers• Layer D1: Network infrastructure layer represented by the general
purpose Internet infrastructure and dedicated network infrastructure• Layer D2: Datacenters and computing resources/facilities, including
sensor network• Layer D3: Infrastructure virtualisation layer that is represented by the
Cloud/Grid infrastructure services and middleware supporting specialised
scientific platforms deployment and operation• Layer D4: (Shared) Scientific platforms and instruments specific fordifferent research areas• Layer D5: Access Infrastructure Layer: Federation infrastructurecomponents, including policy and collaborative user groups support
functionality• Layer D6: Scientific applications and user portals/clients

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

25

SDI move to Clouds• Cloud technologies allow for infrastructure virtualisation andits profiling for specific data structures or to support specificscientific workflows• Clouds provide just right technology for infrastructure virtualisation to
support data sets
• Complex distributed data require infrastructure– Demand for inter-cloud infrastructure• Cloud can provide infrastructure on-demand to support
project related scientific workflows– Similar to Grid but with benefits of the full infrastructure provisioning
on-demand• Software Defined Infrastructure Services– As wider than currently emerging SDN (Software Defined Networks)• Distributed Hadoop clusters for HPC and MPP
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

26

General use case for infrastructure provisioning:
Workflow => Logical (Cloud) InfrastructureEnterprise/Scientific workflow
Storage
Data

SpecialProc 1
Data
Filtering

Input
Data

VisualPresent
SpecialProc 2

Instrum.
Data

Campus A

Data
Archive

Visualisation

Visualisation

CE

User
Group A

Campus B

CE

User
User
User

User
User
User
VR6

Cloud 2 PaaS

VR2

User
Group B

VR7
VR4

VR1

VR5

Resource/
Service
ProviderVR3Enterprise/Project basedIntercloud Infrastructure

Cloud 1 IaaS

Resource/
Service
Provider
CN

CN
CN

CN
CN

CN

Cloud PaaS Provider

CN
CN

CN

CN
CN

Cloud IaaS Provider

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure27General use case for infrastructure provisioning:
Workflow => Logical (Cloud) InfrastructureEnterprise/Scientific workflow
Storage
Data

SpecialProc 1
Data
Filtering

Input
Data

VisualPresent
SpecialProc 2

Instrum.
Data

Campus A

Data
Archive

Visualisation

Visualisation

CE

User
Group A

Campus B

CE

User
User
User

User
User
User
VR6

Cloud 2 PaaS

VR2

User
Group B

VR7
VR4

VR1

VR5

Resource/
Service
ProviderVR3Enterprise/Project basedIntercloud Infrastructure

Cloud 1 IaaS

Resource/
Service
Provider
CN

CN
CN

CN
CN

CN

Cloud PaaS Provider

CN
CN

CN

CN
CN

Cloud IaaS Provider

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure28General use case for infrastructure provisioning:
Logical Infrastructure => Network Infrastructure (1)Resource and Cloud Provider Domains
Cloud 1 IaaS
VR3

VR1

Cloud 2 PaaS

VR5

VR7

Campus A
Infrastructure

Campus B
Infrastructure

VR2

Campus A

VR4

VR6

Cloud Carrier Network Infrastructure

Visualisation

Visualisation

CE

User
Group A

Campus B

CE

User
User
User

User
User
User
VR6

Cloud 2 PaaS

VR2

VR7
VR4

VR1

VR5

Resource/
Service
ProviderDefined as
InterCloud Architecture
Framework (ICAF)

UserGroup B

VR3

Enterprise/Project basedIntercloud Infrastructure

Cloud 1 IaaS

Resource/
Service
Provider
CN

CN
CN

CN
CN

CN

Cloud PaaS Provider

CN
CN

CN

CN
CN

Cloud IaaS Provider

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

29

Big Data and Cloud/Intercloud Research topics• Mapping from scientific workflow to inter-cloud• Data structures and supporting infrastructure• Cloud infrastructure support for Big Data security
and trustworthiness (for generically distributed
scenarios)– Data centric security models– Authenticity, authorisation, delegation– Trust, trustworthy systems, validity– Accounting, auditing– Privacy: Will it change with the new technologies?
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure30

Questions and Discussion

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure31Additional materials• Big Data technologies and foreseen innovations• Trust and trustworthiness

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

32

Big Data: Existing technologies (1)• Big Data storage and database solutions– Google File System, BigTable– New types of databases?

• Big Data computing– MapReduce, Google’s Dremel– MPI and Computer Grids– To be capable to run in a distributed environment closer todata stores and sources• Network for Big Data
– Capacity – Yes– New protocols? – Routing, streaming, load balancing, etc.
optimised for big data, workflow and data structure
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure

33

Big Data: Existing technologies (2)• Social networks and human driven/centriccollaboration and sharing environment– Facebook is a heuristic implementation to human’s crave
to share, socialise and “expand”• Not perfect, to evolve or be overtaken by new more open
technologies to support human’s socialising activity• Current new developments at Facebook– From Like to Open Graph as a behavioral technology– Payment system• Smartphones, tablets and BYOD (Bring Your Own
Device) – to be supported by employer IT– Hub for human/body sensors and wearable devices
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure34

Foreseen Big Data Innovations in 2013
• Cloud-Based Big Data Solutions– Amazon’s Elastic Map Reduce (EMR) is a market leader– Expected new innovative Big Data and Cloud solutions• Real-Time Hadoop– Google’s Dremel-like solutions that will allow real-time queries on Big Data and beopen source• Distributed Machine Learning– Mahout iterative scalable distributed backpropagation machine learning and data
mining algorithm– New algorithms Jubatus, HogWild• Big Data Appliances (also for home)– Raspberry Pi and home-made GPU clusters– Hardware vendors (Dell, HP, etc.) pack mobile ARM processors into server boxes– Adepteva's Parallella will put a 16-core supercomputer into for $99• Easier Big Data Tools– Open Source and easy to use drag-and-drop tools for Big Data Analytics tofacilitate the BD adoption– Commercial examples: Radoop = RapidMiner + Mahout, Tableau, Datameer, etc.

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure35

Predictions for 2013 and as far as 2020•

Main technologies and areas
–
–
–
–

•

Smartphones and tablets
E-Medicine and Healthcare
Big Data
Cloud: shift from private to public

Big Data and Cloud Infographic
http://www.cloudtweaks.com/2013/01/cloud-infographic-big-data-universe/•

Big Data in 2013 by Mike Guattieri, Forrester
–

–
–

–

Firms realize that Big Data means all it's data. Big Data is the frontier of a firms ability
to store, process, and access (SPA) all the data it needs to operate effectively, makedecisions, reduce risks, and create better customer experiences. In 2013, all data is Big
Data.
The algorithm wars begin. Advanced analytics uses advanced statistical, data mining,and machine learning algorithms to dig deeper to find patterns that you can’t see using
traditional BI tools, simple queries, or rules.
Real-time architectures will swing to prominence. Firms will seek out streaming, event
processing, and in-memory data technologies to provide real-time analytics and run
predictive models. Mobile is a key driver because hyper-connected consumers andemployees will require architectures that can quickly process incoming data from all digitalchannels to make business decisions and deliver engaging customer experiences in realtime.
Big Data may be disruptive. This is the year that Time magazine names Big Data as2013 person of the year.

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure36Trust model – Transposition to ICT/SDI domain
Classic Trust definition (“humanised”)
A trusts B to do something(typically Action, Operation, or Assertion)
Trust definition transposed to Trustworthy ICTUser A trusts service B to do an operation OP(or make assertion)with data D on a platform P• Data are associated with the handling policy• Platform is multi-tier, multi-domain and multi-provider• Platform includes Computing, Storage and Network• Data may be stored by 3rd party and belong to domain different to user
21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure37Component of the Trusted/Trustworthy ICTUser Trust domain A
Data

Trust between
domains A and B

Provider Trust domain B

Data stored atuser facilities

Data

Application(Operation)

User

User Client/
System••Platform/
SystemData

ProviderUser Client
trusts provider’s
System/Platform•

Data stored
in Provider
domainTrust(worthiness)
Security

User and Service Provider – the two actors concerned with own Data/Content security and each other
System/Platform trustworthiness
Data may be stored separately but typically belong to user trust domain In clouds it needs to be extended
to distributed, multi-domain and multi-provider environment
Two other aspects of security and trust––

Data stored vs Data accessed/processed
System Idle vs Active (running User session)

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure38

Trustworthiness and Trustworthy system
Trustworthy infrastructure/system definition and analysis must
include more factors• Two or more parties involved into trust relations, typically Users (or user
client/application) and Provider (or server or service/application)• Data with their ownership, privacy, confidentiality level, andapplied/attached handling policy• Computing platform and storage (which however can associated with thegeneral platform)• Underlying communication infrastructure (which however can beassociated with the general platform)• Possible inter-domain issues related to different policies andassurance/provisions

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure39Security in Virtualised Environment (2):Platform, OS/VM, Application and Network + Data
Platform – OS/VM – Container – Application
Multilayer security

Platform – OS/VM – Container – Application
Multilayer security

Virtualisation Platform (Cloud)Virtualisation Platform (Cloud)VPN basedtransport network security

OS/VM
App Container
ApplInter-container TLS/SSL security

OS/VM
App Container
ApplInter-applications e2e security

Metadata
Information/
Registry Service

Session Credentials/Context

Data together with Policy

User/System Credentials, secrets
Platform trust anchor (TPM, hardware secret key)•

Requires multilayer security and trust–•

Security and access control policies may be defined for each layer

Data related components defining access control: metadata, encapsulated policies

21 May 2013, BDDAC2013

Big Data & Sci Data Infrastructure40