Services to establish Data Science Profession:
EDISON Data Science Framework and activities

Yuri Demchenko, EDISON
University of Amsterdam

Third Community Workshop on OpenScience CloudEDISON – Education for Data Intensive
Science to Open New science frontiers
Grant 675419 (INFRASUPP-4-2015: CSA)

7 April 2016
Science Park Amsterdam

Outline•••

Demand for Data Science and data related professions
European initiatives related to Digital Single Market (DSM) and demand todata related competences and skills
EDISON Data Science Framework– From Data Science Competences to Body of Knowledge and Model Curriculum•••

Data Science Competence Framework: Essential competences and skills
Taxonomy: Data Science occupations family
Data Science Body of Knowledge (DS-BoK)••

Knowledge areas and academic disciplines

Further steps

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

Slide_2

Demand for Data Science and data related
professions• McKinsey Global Institute on Big Data Jobs (2011)
http://www.mckinsey.com/mgi/publications/big_data/index.asp– Estimated gap of 140,000 - 190,000 data analytics
skills by 2018

• UK Big Data skills report 2014
– 6400 UK organisations with 100+ staff will have implemented
Big Data Analytics by 2020
– Increase of Big Data jobs from 21,400 (2013) to 56,000 (2017)• IDC Report on European Data Market (2015)– Number of data workers 6.1 mln (2014)– increase 5.7% from 2013
– Average number of data workers per company 9.5 - increase 4.4%
– Gap between demand and supply 509,000 (2014) or 7.5%

• HLEG report on EOSC (2016) identified need for data experts anddata stewards– Estimation: More than 500,000 data stewards (1 per every 20 scientists
or 5% funding)
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession3

Demand for Data Science and Data related professions– EOSC and European RIs• HLEG report on EOSC identified need for data experts and data
stewards– Estimation: More than 500,000 data stewards• 1 per every 20 scientists or 5% funding– Challenges and observation:• Shortage of data experts• Data literacy gap: ‘Valley of death’ between (e-)infrastructure providers and domain
specialists– Requirements: Core data experts need to be trained and their career
perspective significantly improved– Support: Professional data management and Long term data stewardship– Implementation recommendations:••••

I3: Fund a concentrated effort to locate and develop Data Expertise in Europe
I5: Make adequate data stewardship mandatory for all research proposals
II4: Train the data experts to bridge between e-Infra and ESFRI
II5: Assist data stewardship planning and exec tools for all researchers– Open Science Presidency Conference 4-5 April 2016:• Recognised data steward as a profession
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

4

EDISON Framework: Concept, Services, Sustainability

CF-DS

•

DS-BoK

MC-DS

EOEE & ETMplz

Taxonomy & Vocabulary

Edu&Train Inventory

Foundation & Concepts

Services

Sustainability:• Roadmap• Community• Serv/Info Portal• Certification

Biz Model

EDISON Framework components
•
•
•
•
•
•

CF-DS – Data Science Competence Framework
DS-BoK – Data Science Body of Knowledge
MC-DS – Data Science Model Curriculum
Data Science Taxonomy and Scientific Disciplines Classification
EOEE - EDISON Online Education Environment

Background: EU Competence Frameworks and Profiles–––

e-CFv3.0 - European e-Competence framework for IT
CWA 16458 (2012): European ICT Professional Profiles Family Tree
ESCO (European Skills, Competences, Qualifications and Occupations) framework

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

5

Identified Data Science Competence Groups•

Traditional/known Data Science competences/skills groups include
–
–
–

•

Data Analytics or Business Analytics or Machine Learning
Engineering or Programming
Subject/Scientific Domain Knowledge

EDISON identified 2 additional competence groups demandedby organisations[ref] Legacy: NIST BDWG
definition of Data Science– Data Management, Curation, Preservation– Scientific or Research Methods and/vs Business Processes/Operations•

Other skills commonly recognized aka “soft skills” or “social intelligence”–•

All groups need to be represented in Data Science curriculum and training programmes–•Inter-personal skills or team work, cooperativeness

Challenging task for Data Science education and training

Another aspect of integrating Data Scientist into organisation structure–––

General Data Science (or Big Data) literacy for all involved roles and management
Common agreed and understandable way of communication and information/data presentationRole of Data Scientist: Provide such literacy advice and guiding to organisation

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

6

Data Science Competence Groups - Research
Data Science Competence
includes 5 areas/groups
•
•
•
•
•

Data Analytics
Data Science Engineering
Domain Expertise
Data Management
Scientific Methods (or Business
Process Management)

Scientific Methods
•
•
•
•
•
•

Design Experiment
Collect Data
Analyse Data
Identify Patterns
Hypothesise Explanation
Test Hypothesis

Business Operations
•
•
•
•
•

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

Operations Strategy
Plan
Design & Deploy
Monitor & Control
Improve & Re-design7

Data Science Competences Groups – Business
Data Science Competence
includes 5 areas/groups

DOMAIN
EXPERTISE

•
•
•
•
•

DesignOptimisation

RESEARCH

DATA

DATA
ANALYTICS

Scientific Methods

SCIENCEANALYTIC ALGORITHMS
SYSTEMS

Monitoring

Modelling
ENGINEERINGCOMPETENCES

Business Process
Management

Data
Management
Execution

EOSC-3 Wsh - 7 April 2016

Data Analytics
Data Science Engineering
Domain Expertise
Data Management
Scientific Methods (or Business
Process Management)

ScientificMethods
Building Data Science Profession

•
•
•
•
•
•

Design Experiment
Collect Data
Analyse Data
Identify Patterns
Hypothesise Explanation
Test Hypothesis

Business Process
Operations/Stages
•
•
•
•
•

DesignModel/Plan
Deploy & Execute
Monitor & Control
Optimise & Re-design8Identified Data Science Competence Groups
Data Analytics (DA)

1

2

3

4

5
6
7

Data Management/
Curation (DM)
Use appropriate statistical Develop andtechniques on available implement data
data to deliver insights
strategy

DS Engineering (DSE)

Scientific/Research Methods (DSRM)

DS Domain Knowledge(including Business Apps)

Use engineering principles toresearch, design, or develop
structures, instruments,
machines, experiments,
processes, systems, theories,
or technologies

Create new understandings andcapabilities by using the scientific
Understand business andmethod's hypothesis, test, and evaluation provide insight, translatetechniques; critical review; or similar
unstructured businessengineering research and development
problems into an abstract
methods
mathematical framework
Use predictive analytics
Use data to improve existing
Direct systematic study toward a fuller
to analyse big data andservices or develop new
knowledge or understanding of the
discover new relations
fundamental aspects of phenomena and services
Develops specialized data
Develop data models analysis tools to supportof observable facts, and discovers new
including metadata
approaches to achieve goals
executive decision making
Participate strategically andResearch and analyze
Integrate different data
Undertake creative work, making
tactically in financial
complex data sets,source and provide forsystematic use of investigation ordecisions that impact
combine different sources further analysis
experimentation, to discover or revise
Design, build, operatemanagement andand types of data toknowledge of reality, and uses thisrelational non-relationalorganizations
improve analysis.
knowledge to devise new applications
databases
Develop and apply
Recommends business
Develop specialized
related strategic
Develop and maintain computational solutions toanalytics to enablea historical data
domain related problems
objectives andagile decision
repository of analysis using wide range of data
Apply ingenuity to complex problems,
alternatives andmakingdevelop innovative ideas
analytics platforms
implements themCollect and manage
Ability to translate strategies into action
Provides scientific, technical,
different source of dataplans and follow through to completion.
Develop solutions for secureand analytic support services
and reliable data accessto other organisational roles
Analyse multiple data sources
Develop algorithms to analyse Influence the development ofVisualise complex and multiple source of data
for marketing purposesorganizational objectives
variable data.
Prototype new data analytics
Analyse customer data toapplications
identify/optimise customer
relations actions

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

9

Identified Data Science Skills/Experience Groups•

Group 1: Skills/experience related to competences
–
–

•

•

Data Analytics and Machine Learning
Data Management/Curation (including both general data management and scientific data
management)– Data Science Engineering (hardware and software) skills– Scientific/Research Methods– Application/subject domain related (research or business)– Mathematics and Statistics
Big Data Tools and ProgrammingGroup 2: Big Data (Data Science) tools and platforms
Languages– Big Data Analytics platforms• Big Data Analytics platforms– Math & Stats apps & tools• Math& Stats tools– Databases (SQL and NoSQL)• Databases– Data Management and Curation platform• Data/applications visualization• Data Management and Curation– Data and applications visualisation– Cloud based platforms and tools

Group 3: Programming and programming languages and IDE–•

General and specialized development platforms for data analysis and statistics

Group 4: Soft skills or Social Intelligence–

Personal, inter-personal communication, team work (also called social intelligence or soft skills)

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

10

Identified Data Science Skill Groups
Data Analytics andMachine Learning

Data Management/
Curation

Data Science
Engineering (hardware
and software)

Scientific/ Research
Methods

1

Artificial intelligence,machine learning

Design efficientalgorithms for accessing
and analyzing large
amounts of data

2

Machine Learning andStatistical Modelling

Manipulating andanalyzing complex, highvolume, highdimensionality data fromvarying sources
for data improvement

Machine learning
solutions and patternrecognition
techniques

3

4
5

6
7
8

9
10

Application/subject
domain (research or
business)

Interest in data science

Personal/Interpersonalcommunication, teamwork
Communication skills

Big Data solutions andadvanced data mining
tools

Analytical, independent,
critical, curious and focusedon results

Inter-personal intrateam and external
communication

Data Analytics forcommercial purposes

Data models anddatatypes

Multi-core/distributed
software, preferably in a
Linux environment

Confident with large data
sets and ability to identify
appropriate tools andalgorithms

Network of contacts in
Big Data community

Data sources andtechniques for businessinsight and customer
focus

Supervised andunsupervised learning

Handling vast amountsof data

Databases, databasesystems, SQL and NoSQL

Data mining

Experience of working
with large data sets

Statistical analysis
languages and tooling

Flexible analytic approach toachieve results at varying
levels of precision
Exceptional analytical skills

Markov Models,Conditional Random
Fields
Logistic Regression,
Support Vector
Machines
Predictive analysis
and statistics(including Kaggle
platform)(non)relational and (un)structured data

Cloud powered
applications design(Artificial) Neural
Networks

Metadata annotationand management

Statistics

Data citation, metadata,
PID (*)

Recommender or
Ranking system

Mechanism Design
and/or Latent Dirichlet
AllocationGame Theory

Copyright and IPR

Cloud based data
storage and data
management
Data management
planning

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

11

Suggested e-CF Competences for Data Science:
Next eCF Workshop meeting – 14 April 2016
A. PLAN and Design•
A.10* Organisational workflow/processes model definition/formalisation•
A.11* Data models and data structures
B. BUILD: Develop and Deploy/Implement•
B.7* Apply data analytics methods (to organizational processes/data)•
B.8* Data analytics application development•
B.9* Data management applications and tools
15 Data Science Competences proposed•
B.10* Data Science infrastructure deployment
C. RUN: Operate•
C.5* User/Usage data/statistics analysis•
C.6* Service delivery/quality data monitoringcovering different organizational roles andworkflow stages• Data Scientist roles are crossing
multiple org roles and workflow stages

D. ENABLE: Use/Utilise•
D10. Information and Knowledge Management (powered by DS)•
D.13* Data presentation/visualisation, actionable data extraction•
D.14* Support business processes/roles with data and insight (support to D.5, D.6, D.7, D.12)•
D.15* Data management/preservation/curation with data and insight
E. MANAGE•
E.10* Support Management and Business Improvement with data and insight (support to E.5, E.6)•
E.11* Data analytics for (business) Risk Analysis/Management (support to E.3)•
E.12* ICT and Information security monitoring and analysis (support to E.8)

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

12Possible Data Scientist profiles/roles
as extension to CWA16458 (2012)• Data Analyst, Business Analyst– Data Mining– Machine Learning• Digital Librarian, Data Archivist, Data Curator, Data Steward– Data Management related competences• Data Science Engineer/Administrator/Programmer– Data analytics applications development– Scientific programming– Data Science/Big Data Infrastructure development/operation• Data Science Researcher– Data Science research methods– Data models and structures• Data Scientist in subject/research domain• Research e-Infrastructure brings its own specifics to requiredcompetences and skills definition
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

13

Data Science occupations in ESCO taxonomy (1)Professionals
Science andengineering
professionals

Data Science
Professionals

Data Science
professionals notelsewhere classified

Data Scientist

Data Science Researcher(Big) Data Analyst
Data Science (Application)
Programmer
Business Analyst
Database and network
professionals

Information andcommunications
technology
professionals

Data Science
technology
professionals

Large scale (cloud) data
storage designers andadministrators

Large scale (cloud) database
designer*)

Database designers andadministrators

Large scale (cloud) databaseadministrator*)

Database and network
professionals notelsewhere classified

Scientific databaseadministrator*)

Data handlingprofessionals notelsewhere classified

Digital Librarian

Data Archivist
Data Steward
Data curator
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

14

Data Science occupations in ESCO taxonomy (2)Technicians and associate professionals
Science andengineering associate
professionals

Data Science
Technology
Professionals

Data Infrastructureengineers andtechnicians

Big Data facilities Operators

Large scale (cloud) data
storage operators
Database and network
professionals notelsewhere classified

Scientific database operator*)

Managers
Production andspecialised services
managers

Data Science/Big Data
Infrastructure Managers

Data Science/Big Data
Infrastructure Manager
Research InfrastructureManagers

RI Manager
RI Data storage facilities
manager

Clerical support workers
General andkeyboard clerks
Data handling
support workers(alternative)

Data and information
entry and access

Digital Archivists andLibrarians

Digital Librarian

Data Archivist
Data Steward
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

Data curator

15

Data Science or Data Management Support Group:
Organisational structure and staffing

•
•
•
•
•

Manager
Data Science Architect
Data Analyst
Data Science Application programmer
Data Infrastructure/facilities administrator/operator: storage,
cloud, computation• Data stewards

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

16

Data Science or Data Management Support Group:
Organisational structure and staffing - EXAMPLE

•
•
•
•
•

Manager (1)Data Science Architect (1)Data Analyst (2)Data Science Application programmer (3)Data Infrastructure/facilities administrator/operator: storage,
cloud, computation (1-2)• Data stewards (5-10)
Group of 14-20 specialists

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

17

Education and Training• Foundation and methodological base
– Data Science Body of Knowledge (DS-BoK)• Taxonomy and classification of Data Science related scientificsubjects– Instructional methodologies and teaching models• Platforms and environment– Virtual labs, datasets, developments platforms– Access control and accounts/identity management– Online education environment and courses management• Services– Individual profiling tools (competence assessment)– Knowledge evaluation tools– Certifications and training– Courses catalog and repository: Education and training
marketplace
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

18

Data Science Body of Knowledge (DS-BoK)
DS-BoK Knowledge Area Groups (KAG)•••

KAG1-DSA: Data Analytics group including Machine Learning, statisticalmethods, and Business Analytics
KAG2-DSE: Data Science Engineering group including Software andinfrastructure engineering
KAG3-DSDM: Data Management group including data curation,
preservation and data infrastructure
KAG4-DSRM: Scientific/Research Methods group
KAG5-DSBP: Business process management group•

Data Science domain knowledge to be defined by related expert groups••

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

19

KAG3-DSDM: Data Management group: data curation,
preservation and data infrastructure
DM-BoK version 2 “Guide for
performing data management”– 11 Knowledge Areas(1) Data Governance,(2) Data Architecture,(3) Data Modelling and Design,(4) Data Storage and Operations,(5) Data Security,(6) Data Integration andInteroperability,(7) Documents and Content,(8) Reference and Master Data,(9) Data Warehousing and BusinessIntelligence,(10) Metadata,(11) Data Quality

EOSC-3 Wsh - 7 April 2016

Other Knowledge Areas motivatedby RDA, European Open Data
initiatives, European Open Data
Cloud(12) PID, metadata, data registries(13) Data Management Plan
(14) Open Science, Open Data,
Open Access, ORCID(15) Responsible data use

Building Data Science Profession

20

Topics considered for the Data Management(Literacy) Training – Working draft
A. Use cases for data management and stewardship•
Preserving the Scientific Record
B. Data Management elements (organisational and individual)•
Goals and motivation for managing your data•
Data formats•
Creating documentation and metadata, metadata for discovery•
Using data portals and metadata registries•
Tracking Data Usage•
Handling sensitive data•
Backing up your data•
Data Management Plan (DMP) - to be a part of hands on session
C. Responsible Data Use Section (Citation, Copyright, Data Restrictions)
D. Open Science and Open Data (Definition, Standards, Open Data use and reuse, open government data)•
Research data and open access•
Repository and self- archiving services•
ORCID identifier for data•
Stakeholders and roles: engineer, librarian, researcher•
Open Data services: ORCID.org, Altmetric Doughnut, Zenodo
E. Hands on:a) Data Management Plan design–
Why Create a Data Management Plan?
–
Elements of a Data Management Plan–
Organization and Standards–
Data management plan checklistb) Metadata and toolsc) Selection of licenses for open data and contents (Creative Common and Open Database License)

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

21

Further Steps•

Define a taxonomy and classification for DS competences and skills as a basis for moreformal CF-DS definition–•

Closer look at skills, tools and platforms

Run surveys for target communities
https://www.surveymonkey.com/r/EDISON_project_-_Defining_Data_science_profession– Plan a number of key interviews, primarily experts and top executives at universities and companies•

Proceed with suggested e-CF3.0 extensions and participate in the next e-CF meetings
–

•
•
•

Provide feedback and contribution to ESCO with the definition of the Data Science
professions family
Suggest ACM2012 Classification extensions and contact ACM people
Provide input to DS-BoK definition following from CF-DS–•

Talk to national e-CF bodies or adopters if available

Link/Map to taxonomy of academic and educational and training courses

Create open community forum to collect contribution––

CF-DS and DS-BoK documents are on public comments available from EDISON website
http://www.edison-project.eu/data-science-competence-framework-cf-ds
http://www.edison-project.eu/data-science-body-knowledge-ds-bok
Start related Social Network groups to promote already obtained results and obtain feedback andcommunity contribution

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

22

Discussion• Questions• Observations• Survey: Invitation to participate

https://www.surveymonkey.com/r/EDISON_project_-_Defining_Data_science_profession
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

23

Data Scientist: New Profession and Opportunities
McKinsey Institute on Big Data Jobs (2011)
http://www.mckinsey.com/mgi/publications/big_data/index.asp•

There will be a shortage of talent necessary for organizations to take advantage
of Big Data.
––

By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep
analytical skills as well as
1.5 million managers and analysts with the know-how to use the analysis of big data to makeeffective decisions

SOURCE:US Bureau of Labor Statistics; US Census; Dun & Bradstreet; company interviews; McKinsey analysis
EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

24

EXAMPLE: Use of e-CF3.0 for Defining Profile of
RI Technical (part of RDA IG-ETRD work)
A. PLAN and DESIGN

D. USE: UTILISE (ENABLE)

A.2. Service Level Management
A.3. Product / Service Planning
A.5. Application Design
A.4. Architecture Design

D.1. Scientific Applications Integration (on running RI)
D.5. Data collection and preservation
D.4. New requirements and change Identification
D.6. Education and Training Provision

Additional

A.6. Sustainable Development
A.7. Innovating and Technology Trend Monitoring
A.8. Business/Research Plan Development and Grant
application
A.1. RI and Research Strategy Alignment
B. BUILD: DEVELOP and DEPLOY/IMPLEMENT

B.1. Application Development (Reqs Engineering,
Function Specs, API, HCI)
B.2. Component Integration
B.3. Testing (RI services and Scientific Apps)
B.4. Solution/Apps Deployment
Additional

B.5. Documentation Production
B.6. Systems Engineering (DevOps)
C. OPERATE (RUN)

C.1. User Support
C.2. Service Delivery
C.3. Problem Management

Additional

D.2. Information Security Strategy Development
D.3. RI/ICT Quality Strategy Development
D.7. Purchasing/Procurement
D.8. Contract Management
D.9. Personnel Development
D.10. Dissemination and outreach
E. MANAGE

E.1. Overall RI management (by systems andcomponents)
E.5. Information/Data Security Management
Additional

E.6. Data Management (including planning and lifecycle
management, curation)
E.4. RI Security and Risk/Dependability Management
E.2. Project and Portfolio Management
E.3. ICT Quality Management and ComplianceE.7. RI/IS Governance

Additional

C.4. Change Support (Upgrade/Migration)

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession25

Data Scientist and Subject Domain Specialist•

Subject domain components
–
–
–
–
–

•

Model (and data types)
Methods
Processes
Domain specific data and presentation/visualization methods (?)
Organisational roles and relations

Data Scientist is an assistant to Subject Domain Specialists– Translate subject domain Model, Methods, Processes into abstract data driven
form– Implement computational models in software, build required infrastructure andtools– Do (computational) analytic work and present it in a form understandable tosubject domain– Discover new relations originated from data analysis and advice subject domain
specialist– Present/visualise information in domain related actionable way– Interact and cooperate with different organizational roles to obtain data anddeliver results and/or actionable data

EOSC-3 Wsh - 7 April 2016

Building Data Science Profession

26

Data Science and Subject Domains
Data Science domain components
Data structures &databases/storage
Visualisation• Abstract data driven
math&compute models• Data Analytics
methods• Data and Applications
Lifecycle Management

Crossorganisational
assistive role

Data Scientist functions is to translate between two domains

Domain specific components
Domain specificdata & presentation(visualization)

EOSC-3 Wsh - 7 April 2016

• Models (and data types)• Methods• Processes

Building Data Science Profession

Organisationalroles

27