Big Data Infrastructure Technologies for
Data Analytics
andData Science Projects Operationalisation

Special Session on Big Data
IDAACS2021 22-25 September 2021
Dr. Yuri Demchenko
University of Amsterdam

Outline• Big Data Reference Architecture and functional components• Data Science development process– Data Science/Data Mining Process models, Dataflow/Data Lifecycle• Operationalising Data Science Analytics and ML: DataOps, MLOps and platforms– Data Science and ML pipeline• Case study: Data Science and MLOps with Azure• Further to data driven projects operationalization with Site Reliability Engineering (SRE)• Summary

IDAACS2021

Big Data and Data Science Projects Operationalisation2

Goal of this Research and Talk
• Identify evolution and trends in Big Data technologies and Data
Science projects move from desk development to operational stage• Why it is needed and what is the benefit?
When trends are identified (among variety of technologies and offers) it is easy tofollow and understand further developments in technologies andindustrial/business applications

IDAACS2021

Big Data and Data Science Projects Operationalisation

3

Multiple aspects of Big Data andRelation to Data ScienceBig Data is a complex of technologies toenable handling of Big Data (storage,processing, transfer, security)
Data Science is an inter-disciplinary field thatuses scientific methods, processes, algorithms
and systems to extract knowledge and insightsfrom structured and unstructured data.
• Data Science vs Data Analytics vs StaticalAnalysis
IDAACS2021

Big Data and Data Science Projects Operationalisation

4

Big Data and Multiple Sources of Data•••

Social MediaInternet
Communication, voice, video

IDAACS2021•••

ScienceIndustrial data
IoT

Big Data and Data Science Projects Operationalisation•Commonly blending DataAnalytics with open andsocial media data•

IoT is taking a front stage indriving Big Data
infrastructure and Data
Analytics applications•Data Spaces demand a fullecosystem with data centric
infrastructure and data driven
applications

5

NIST Big Data Working Group (NBD-WG) and ISO/IEC JTC1 Study
Group on Big Data (SGBD)•••NIST Big Data Working Group (NBD-WG) is leading the development of the Big Data Technology Roadmap http://bigdatawg.nist.gov/home.php– Built on experience of developing the Cloud Computing standards
Published as NIST Special Publication 1500 Volumes 1-7 in 2015
New revision V3 published 2018 - https://bigdatawg.nist.gov/V3_output_docs.phpVolume 1: Definitions
Volume 2: Taxonomies
Volume 3: Use Case & Requirements
Volume 4: Security & Privacy
Volume 5: Architecture White Paper
Volume 6: Reference Architecture
Volume 7: Standards Roadmap
Volume 8: Reference Architecture Interface
Volume 9: Modernization and Adoption

The Big Data Paradigm consists of thedistribution of data systems acrosshorizontally-coupled independent resources toachieve the scalability needed for theefficient processing of extensive datasets.

• NBD-WG defined 3 main components of the new technology:– Big Data Paradigm– Big Data Science and Data Scientist as a new profession– Big Data Architecture
IDAACS2021

Big Data and Data Science Projects Operationalisation

6NIST Big Data Reference Architecture (2018)
Main components of the Big Data ecosystem• Data Provider• Big Data Applications Provider• Big Data Framework Provider• Data Consumer• Service Orchestrator

Big Data Lifecycle and Applications Provider
activities•••••

Collection
Preparation
Analysis and Analytics
Visualization
AccessBig Data Ecosystem includes all componentsthat are involved in the big data production,processing, delivery, and consuming

IDAACS2021

Big Data and Data Science Projects Operationalisation[ref] Volume 6: NIST Big Data Reference Architecture.
http://bigdatawg.nist.gov/V1_output_docs.php
7

CCI and UvA Research on Big Data Technologies and Scientific DataInfrastructure – Journey driven by Technology Development• 2010-2015 Starting Cloud Computing research and contribution to NIST Big Data WG on SP 1500
first edition 2015
• Numerous projects related to e-Infrastructure and European Research Infrastructures– EGEE, GEANT, EOSC FAIR, ENVRI, SLICES-RI, AMDeX• Research on Big Data Infrastructure technologies and data centric applications– Platform Engineering and adopting Digital Platform Architecture model• Data Spaces and IDSA Reference Architecture, Interoperability Framework– Data Spaces optimization for Trustworthiness, Sustainability, Societally Responsible Development– Data sharing and Data Exchanges• DataOps: DevOps for Data Science projects – Operationalisation of Data Science projects

IDAACS2021

Big Data and Data Science Projects Operationalisation

8

Data Science Project Management••••Data Science development process
Data Science/Data Mining Process models, Data Flow/Data Lifecycle
Operationalising Data Science Analytics and ML: DataOps, MLOps and platforms
Case study: Azure Analytics stack and MLOps platform

IDAACS2021

Big Data and Data Science Projects Operationalisation

9

Data Science Development Process
Data Science process is dealing with the data pipelines that include stages:• Collecting data from multiple sources, also blending process or business data with external datasuch as environmental data or social media data that can be obtained via WebAPI or web scraping• Working with data including data preparation, cleaning, filtering, and reformatting for modelingneeds• Combing datasets by joining on common attributes, consolidating attributes, build tabular data
structure (such as used in popular analytics programming languages R, python, scala)• Feature engineering, algorithm selection• Testing before production and continuously validating the model during production, in particular,detecting drift in predictive models• Implement changes and deploy an updated model.

• Data Science process is linked to Data Lifecycle and Dataflow
IDAACS2021

Big Data and Data Science Projects Operationalisation

10

Data Lifecycle/Transformation Model•
Multiple Data Models and structures•
Data Variety and Variability•
Semantic Interoperability

Data Model (1)Data Model (2)Data (inter)linking•PID/OID•
Identification•Privacy, Opacity

Data Model changes
along data lifecycle orevolution (Variability)
Data provenance(lineage) is adiscipline to track alldata transformations
along their lifecycle
Identifying and linking
data•

Data Model (3)Data Model (4)•Data Storage (Big Data capable, e,g. Data Lake)

DataSource

DataCollection&Registration

Data
Filter/Enrich,Classification

Data
Analytics,
Modeling,
Prediction

Data
Delivery,
Visualisation

Consumer
applications–––

Persistent data/object
identifiers (PID/OID)
Traceability vs Opacity
Referral integrity

Data repurposing,
Analytics re-factoring,
Secondary processing

IDAACS2021

Big Data and Data Science Projects Operationalisation

11

Data Science Project/Research is based on Dataflow/Lifecycle
Model selection and Training

Operation, Monitoring
and Optimisation
Operation &Serving•••

Each phase – data preparation, model training and evaluation, and model deployment – operates on its own data set. All thesedata sets need to be isolated but linked. The pollution of data sets across the data science assembly line is one of the mostfrequent mistakes in model production.
The data science project is typically starts with some historical data or sample dataset can be somewhere in existing repositories.
Data preparation may also include connecting external data sources – data blending[Ref] https://www.knime.com/blog/analytics-and-beyond
IDAACS2021

Big Data and Data Science Projects Operationalisation

12

Data Science Process Models and Model Formats• Data Science process models– CRISP-DM, CRoss-Industry Standard Process for Data Mining– ASUM, Analytics Solutions Unified Method (IBM)– TDSP, Team Data Science Process (Microsoft)– KNIME Model Factory (KMF)• Data Analytics Model Formats••••

From model creation todeployment on Big
Data/cloud platform toProduction

Predictive Models Markup Language (PMML)
Portable Format for Analytics (PFA)ONNX (Open Neural Network Exchange)
TensorFlow Model

IDAACS2021

Big Data and Data Science Projects Operationalisation

13CRISP DM Process: Processes and Data Lifecycle(Cross-Industry Standard Process for Data Mining)

Global view: CRISP-DMOperation
Monitoring

Business
Understanding

Cloud based Big Data
Infrastructure
Deployment

Data access:API, SQL/NoSQL

Data Understanding

Data Preparation

Validation

The Predictive Model Markup
Language (PMML)IDAACS2021

Modelling
Evaluation
Deployment => Operation– Process monitoring

Data

Evaluation– Data Validation•••Modeling

Model representation:
PMML

Cross Industry Standard Process
for Data Mining (CRISP-DM)
model and stages• Business understanding• Data Understanding• Data preparation

All stages are iterative with thegoal to achieve effectiveness forbusiness decision making

Model generation:
SQL/MM, ML

CRISP-DM historically published 1st version in 1999
Big Data and Data Science Projects Operationalisation

14

MLOps and DataOps: DevOps for ML and Data Analytics
MLOps and DataOps are extension of DevOps to manage Data Analytics and Machine Learning data
flow and process.
• DevOps is the practice of operations and development engineers participating together in the entire
service lifecycle, from design through the development process to production support.
• Develop – Build – Deploy – Operate• Cloud is an enabler for DevOps processes
DataOps, MLOps is about operationalizing ML and Data Analytics• Different nature of processes at the stage of ML model development• Benefit from the DevOps processes and culture• Learn from DevOps experience

IDAACS2021

Big Data and Data Science Projects Operationalisation

DevOps Essentials (fromSoftware Engineering)• Better Software, Faster time tomarket• Synergy of Development andOperation• Covers the *entire* ApplicationLifecycle• Continuous Improvement inCI/CD cycle
15Big Data Platform: Research, Development and Production• Big Data require cloud scale due to Big Data data gravity: Compute,Storage and Network optimization + Energy consumption• All major cloud providers offer Big Data platforms and services– Amazon Web Services (AWS)– Microsoft Azure– Google Cloud Platform (GCP)• Specialised Data Analytics providers and vendors

IDAACS2021

Big Data and Data Science Projects Operationalisation

16

The Top 11 (2020) and Top 14 (2021) Big Data Analytics Software [ref]
2020

2021

[ref] https://www.g2.com/categories/big-data-analytics?tab=highest_rated
IDAACS2021

Big Data and Data Science Projects Operationalisation

17

Hadoop Ecosystem: Platform for Big Data Analytics• Basically Open Source withmultiple commercial service
platforms• Amazon EMR – Elastic Map
Reduce– Includes Spark• Azure HDInsight• Azure Databricks and Delta Lake(from Databricks)• Cloudera/Hortonworks•IDAACS2021

Big Data and Data Science Projects Operationalisation

Data Lakes technology is based on
HDFS (Hadoop Distributed File System)

18

Google, AWS, Azure Big Data Stacks

GCP

IDAACS2021

AWS

Big Data and Data Science Projects Operationalisation

Azure

19

Google Cloud Platform

GCP● Machine Learning embedded across most products• Multiple Tensorflow ML models in use– Portable TensorFlow models● Key models exposed via APIs (DemocratizingMachine Learning)–IDAACS2021

Cloud Video Intelligence API

Big Data and Data Science Projects Operationalisation––––•

Cloud Vision API
Cloud Natural Language API
Cloud Translation API
Cloud Speech API

Acquired Kaggle in 2017 - Data Science Enthusiasts

20

Amazon Web Services (AWS)AWSIDAACS2021

Big Data and Data Science Projects Operationalisation

21

Case Study: Azure Data Science and Analytics Stack• Empowering Data Science Process with DevOps and MLOps• Business oriented with good integration with Biz Analytics platforms• Azure Data Lake and Delta Lake
• Azure HDInsight and Azure Databricks Spark• Azure DevOps and MLOps

IDAACS2021

Big Data and Data Science Projects Operationalisation

22

What is Azure Data Lake (ADL) Store?
A highly scalable, distributed, parallel file system in the cloud specifically designed to work with multiple
analytic frameworks

Devices

ADL Analytics
HDInsight

ADL StoreRSpark
Machine Learning•••

Unstructured
Semi-structured
Structured•••

IDAACS2021

Unlimited account size TB, PB
Individual files size from gigabytes
to petabytesNo limits to scale

Big Data and Data Science Projects Operationalisation

23

Azure Data Lake: Part of Cortana Analytics Suite

Information
managementBusinessapps

Big data stores

Machine Learning
and Analytics

Dashboards andvisualizations
Power BI

Azure
Data Factory

Azure
Machine Learning

Azure

SQL Data Warehouse

Personal digitalassistantCortana

Customapps

Azure
Stream Analytics

Azure
Data Catalog

Azure
HDInsight (Hadoop)

Azure
Data Lake Store

Azure
Data Lake Analytics

Azure
Event HubSensors
and devices

DATA

IDAACS2021• Azure Stack for Azure
services deployment
at premises
People• Fully hybrid Data
Lake Analytics

Perceptual intelligence
Face, visionSpeech, text

Business scenarios
Recommendations,customer churn,
forecasting

INTELLIGENCEBig Data and Data Science Projects Operationalisation

Automatedsystems

ACTION• Automatic loadbalancing andoutsourcing to Azure
cloud24

Azure MLOps: Data Science Workflow
PrepareDeployExperiment…

PrepareDataFeatureengineering

Model training& testing

Data Engineer

Data Scientist

ML Engineer

Register &Manage Model

Package &Validate
Model

Deploy Service
Monitor ModelAzure Data Science and Machine Learning Environment
https://docs.microsoft.com/en-us/azure/architecture/solution-ideas/articles/azure-databricks-datascience-machine-learning• Optimized Spark engine• Machine learning run time
– PyTorch, TensorFlow, and scikitlearn• MLflow• Choice of language– Python, Scala, R, Spark SQL and.Net• Collaborative notebooks• Delta lake
• Native integrations with Azure
services• Interactive workspaces• Enterprise-grade security• Production-ready with Azure
DevOps– Ecosystem integrations for CI/CDand monitoring.
IDAACS2021

Big Data and Data Science Projects Operationalisation

26

Azure – Data Science Virtual Machine (DSVM)
https://azure.microsoft.com/en-us/services/virtual-machines/data-science-virtual-machines/•Pre-Configured virtual machines in the cloudfor Data Science and AI Development– Python, R, ONNX, + Microsoft AutoML–•

https://docs.microsoft.com/en-us/azure/machinelearning/data-science-virtual-machine/tools-included

Standard Configuration
Standard_DS3_v2 – 4 vcups – 1 GiB– @ 328.12 USD/mo••

Expected to be used as a virtual desktop
DSVM configuration– Demchmin :: BDIT****ndows

IDAACS2021

Big Data and Data Science Projects Operationalisation27

Site Reliability Engineering: Gluing together Data Analytics andOperation• Further shift to Operation stage in DevOps and service-oriented
customer-facing approach• Empowered by Data Analytics and DataOps• To facilitate ML/AI operationalisation

IDAACS2021

Big Data and Data Science Projects Operationalisation

28

SRE Pillars according to Capgemini [ref]ML/AI enabled monitoring as
a way to manage complex
customer and environment KPI:
Trustworthiness, Sustainability andResponsibility

KPIs are not only numerical

ML/AI enabled analysis
and optimisation[ref] https://www.capgemini.com/2020/08/site-reliability-engineering-2/
IDAACS2021

Big Data and Data Science Projects Operationalisation

29

SRE and Data Analytics Engagement
What are benefits of suchfusion/integration• Continuous model improvement– Essential for operational AI and DigitalTwins• Monitoring and optimization on bothmeasurable and assessable/ranked
KPI[ref] https://sre.google/workbook/engagement-model/
IDAACS2021

Big Data and Data Science Projects Operationalisation

30

Discussion and Questions• Recent trends to Data Analytics and ML Operationalisation• Facilitated by digitalization and easy data concentration• Role of and dependence on Big Data platform (cloud based)

IDAACS2021

Big Data and Data Science Projects Operationalisation31

From DataOps, DevOps to ModelOps
https://www.techzine.nl/blogs/data/432735/na-dataops-devops-nu-ook-modelops/• What is DataOps by IBM - https://www.ibm.com/nl-en/analytics/dataops• DataOps: Industrializing Data and Analytics Strategies for Streamlining theDelivery of Insights
https://s3.amazonaws.com/eckerson/content_assets/assets/000/000/195/original/DataOPS.pdf?1534882627• DataOps: Simplify Managing Complex Data Environments
https://www.sentryone.com/dataops-overview• DataOps is NOT Just DevOps for Data
https://medium.com/data-ops/dataops-is-not-just-devops-for-data-6e03083157b7

IDAACS2021

Big Data and Data Science Projects Operationalisation

32