Practical Machine Learning 101
An end-to-end machine learning project

Justin Looye – IIJ Research Laboratory

Tech Trend Talk: Practical ML 101

2024-07-09

Takeaway from today’s talk - 今日の話から学べること■

Overview of machine learning - 機械学習の概要■

Machine learning pipeline in practice with an end-to-end project

エンドツーエンドプロジェクトによる実践的な機械学習パイプライン

zdataset.com: A Guide to Machine Learning Pipelines and Orchest■

Popular python tools – 人気のある Python ツール
Pandas, Scikit-Learn, MLflow, FastAPI

Tech Trend Talk: Practical ML 101

2024-07-09

2/22

Material for today■“Hands-On MachineLearning with Scikit-Learn,Keras, and TensorFlow 3rdedition” by Aurélien Géron(2022)

Tech Trend Talk: Practical ML 101

2024-07-09

3/22

Material for today■■“Hands-On Machine
Learning with Scikit-Learn,Keras, and TensorFlow 3rdedition” by Aurélien Géron(2022)
Well written, easy to follow
with notebooks

Tech Trend Talk: Practical ML 101

2024-07-09

3/22

Material for today■■■“Hands-On Machine
Learning with Scikit-Learn,Keras, and TensorFlow 3rdedition” by Aurélien Géron(2022)
Well written, easy to follow
with notebooks
Low priors

Tech Trend Talk: Practical ML 101

2024-07-09

3/22

Material for today■■“Hands-On Machine
Learning with Scikit-Learn,Keras, and TensorFlow 3rdedition” by Aurélien Géron(2022)
Well written, easy to follow
with notebooks■

Low priors■

Permissive apache license
Tech Trend Talk: Practical ML 101

2024-07-09

3/22

Why using Machine Learning: a paradigm shift■

Let’s consider a spam filter:

Tech Trend Talk: Practical ML 101

2024-07-09

4/22

Why using Machine Learning: a paradigm shift■

Let’s consider a spam filter:1) Decide what a spam looks like:contains “free”, “credit card”,…
2) Write a detection algorithm based on
rules of 1)
3) Repeat 1) and 2) until successfully
filtered

Tech Trend Talk: Practical ML 101

2024-07-09

4/
22

Why using Machine Learning: a paradigm shift■

Let’s consider a spam filter:1) Decide what a spam looks like:contains “free”, “credit card”,…
2) Write a detection algorithm based on
rules of 1)
3) Repeat 1) and 2) until successfully
filtered1) Get emails labeled as spam or ham2) Select a performance criterion(Accuracy? Recall? …)
3) Let a ML model find the rules.
Software carpentry https://wvuhpc.github.io/2019-Machine-Learning/01-introduction/index.html

Tech Trend Talk: Practical ML 101

2024-07-09

4/22

Why using Machine Learning: a paradigm shift■

Let’s consider a spam filter:1) Decide what a spam looks like:contains “free”, “credit card”,…
2) Write a detection algorithm based on
rules of 1)
3) Repeat 1) and 2) until successfully
filtered

1) Get emails labeled as spam or ham2) Select a performance criterion(Accuracy? Recall? …)
3) Let a ML model find the rules.
Software carpentry https://wvuhpc.github.io/2019-Machine-Learning/01-introduction/index.html■

Advantages of ML:–

Automated decision making (easy to update)–No human priors (unicode abuse)Tech Trend Talk: Practical ML 101

2024-07-09

4/
22

Machine learning problems in the real world

Tech Trend Talk: Practical ML 101

2024-07-09

5/
22

Machine learning problems in the real world
Supervised learning■

Trafficforecasting■House priceprediction■■

E-mail spam
detection
Image
recognition

Tech Trend Talk: Practical ML 101

2024-07-09

5/
22

Machine learning problems in the real world
Supervised learning■

Trafficforecasting■

House priceprediction■■

E-mail spam
detection
Image
recognition

Unsupervised learning■

Credit cardfraud
detection■■

Recommendersystems■

Datavisualization
Compression

Tech Trend Talk: Practical ML 101

2024-07-09

5/
22

Machine learning problems in the real world
Supervised learning

Unsupervised learning

Reinforcement learning

Nisha Arya @ EJable.com: Three Types of Machine Learning■

Trafficforecasting■

House priceprediction■■

E-mail spam
detection
Imagerecognition■■

Credit cardfraud
detection■

Recommendersystems■

Datavisualization■

Autonomous agents■

Robots■Optimization

Compression

Tech Trend Talk: Practical ML 101

2024-07-09

5/22

Basic challenges of ML

Tech Trend Talk: Practical ML 101

2024-07-09

6/22

Basic challenges of ML■

Insufficient quantityoftraining data

Tech Trend Talk: Practical ML 101

2024-07-09

6/22

Basic challenges of ML■■

Insufficient quantityoftraining data
Nonrepresentative
training data

Wikipedia: Martin Grandjean (vector), McGeddon (picture), Cameron Moll (concept)

Tech Trend Talk: Practical ML 101

2024-07-09

6/22

Basic challenges of ML■■■

Insufficient quantityoftraining data
Nonrepresentative
training data
Poor-Quality data
Wikipedia: Martin Grandjean (vector), McGeddon (picture), Cameron Moll (concept)

Tech Trend Talk: Practical ML 101

2024-07-09

6/22

Basic challenges of ML■■

Insufficient quantityoftraining data
Nonrepresentative
training data■

Poor-Quality data■

Irrelevant features
Wikipedia: Martin Grandjean (vector), McGeddon (picture), Cameron Moll (concept)

Tech Trend Talk: Practical ML 101

2024-07-09

6/22

Basic challenges of ML■■

Insufficient quantity oftraining data
Nonrepresentative
training data■

Poor-Quality data■

Irrelevant features■■

Overfitting
the training data
Underfitting
the training data

Jeeva Saravanan @ Medium: How to Evaluate your Machine Learning model

Tech Trend Talk: Practical ML 101

2024-07-09

6/22

End-to-end ML project■■

We are a recently hired data
scientist in a real estate company
The company want to be able topredict district median housevalues

Tech Trend Talk: Practical ML 101

2024-07-09

7/22

End-to-end ML project■■■

We are a recently hired data
scientist in a real estate company
The company want to be able topredict district median housevalues

Company gives us a dataset of district median house values
IDFeatures (X)Tech Trend Talk: Practical ML 101

Target (Y)2024-07-09

7/
22

Project overview

1)Look at the big picture – frame the problem2)Get the data
3)Discover and visualize data to gain insights4)Prepare data for ML algorithms5)Select a model and train it6)Launch, monitor and maintain

Tech Trend Talk: Practical ML 101

2024-07-09

8/
22

Frame the problem

Tech Trend Talk: Practical ML 101

2024-07-09

9/22

Frame the problem

1)Supervised, unsupervised, reinforcement
learning?

Tech Trend Talk: Practical ML 101

2024-07-09

9/22

Frame the problem

1)Supervised, unsupervised, reinforcement
learning?
2)Task? Regression, classification, clustering,...?

Tech Trend Talk: Practical ML 101

2024-07-09

9/22

Frame the problem

1)Supervised, unsupervised, reinforcement
learning?
2)Task? Regression, classification, clustering,...?
3)Performance criterion?

Tech Trend Talk: Practical ML 101

2024-07-09

9/22

Frame the problem

1)Supervised, unsupervised, reinforcement
learning?
2)Task? Regression, classification, clustering,...?
3)Performance criterion?
4)Baseline? Human-level performance? Experts?

Tech Trend Talk: Practical ML 101

2024-07-09

9/22

Tool #1: Pandas for handling data■

Open-source Python library providing highperformance, easy-to-use data structures andanalysis tools

Tech Trend Talk: Practical ML 101

2024-07-09

10 /
22

Tool #1: Pandas for handling data■■

Open-source Python library providing highperformance, easy-to-use data structures andanalysis tools
Key feature: DataFrame object for tabular data
manipulation with integrated indexing

Tech Trend Talk: Practical ML 101

2024-07-09

10 /22

Tool #1: Pandas for handling data

Open-source Python library providing highperformance, easy-to-use data structures andanalysis tools■
Key feature: DataFrame object for tabular data
manipulation with integrated indexing■
Core features:■–Data manipulation–

Data cleaning–

File handling–Time series
Tech Trend Talk: Practical ML 101

2024-07-09

10 /
22

Tool #1: Pandas for handling data■

Open-source Python library providing high-performance, easy-touse data structures and analysis tools■

Key feature: DataFrame object for tabular data manipulation withintegrated indexing■

Core features:■–

Data manipulation–

Data cleaning–

File handling–Time series

Use case in ML:–

Data preparation–

Exploratory data analysis and visualization
Tech Trend Talk: Practical ML 101

2024-07-09

10 /
22

Tool #2 : scikit-learn for Machine learning
■

■

■

■

Simple and efficient
tools for predictive
data analysis
Accessible toeverybody, andreusable in various
contexts
Built on Numpy, SciPy
and matplotlib
Open source,commercially usable –BSD license
Tech Trend Talk: Practical ML 101

2024-07-09

11 /22

Project overview

1)Look at the big picture – frame the problem2)Get the data
3)Discover and visualize data to gain insights4)Prepare data for ML algorithms
5)Select a model and train it6)Launch, monitor and maintain

Tech Trend Talk: Practical ML 101

2024-07-09

12 /22

Get the data

Tech Trend Talk: Practical ML 101

2024-07-09

13 /
22

Get the data■

Parse the data

Tech Trend Talk: Practical ML 101

2024-07-09

13 /
22

Get the data■

Parse the data■

Take a quick look
–

Type–

Missing Values

Tech Trend Talk: Practical ML 101

2024-07-09

13 /
22

Get the data■

Parse the data■

Take a quick look
–

Type–

Missing Values–

Basic stats

Tech Trend Talk: Practical ML 101

2024-07-09

13 /
22

Get the data■

Parse the data■

Take a quick look
–

Type–

Missing Values–

Basic stats

Tech Trend Talk: Practical ML 101

2024-07-09

13 /
22

Get the data■

Parse the data■

Take a quick look
–Type–Missing Values–

Basic stats

Tech Trend Talk: Practical ML 101

2024-07-09

13 /22

Get the data■

Parse the data■

Take a quick look■

–Type–

Missing Values–

Basic stats

Train-test split

Tech Trend Talk: Practical ML 101

2024-07-09

13 /
22

Project overview

1)Look at the big picture – frame the problem2)Get the data
3)Discover and visualize data to gain insights4)Prepare data for ML algorithms
5)Select a model and train it6)Launch, monitor and maintain

Tech Trend Talk: Practical ML 101

2024-07-09

14 /
22

Discover and visualize data to gain insights

Tech Trend Talk: Practical ML 101

2024-07-09

15 /
22

Discover and visualize data to gain insights■

Visualize
geographicaldata

Tech Trend Talk: Practical ML 101

2024-07-09

15 /
22

Discover and visualize data to gain insights■■

Visualize
geographicaldata
Look forcorrelation

Tech Trend Talk: Practical ML 101

2024-07-09

15 /
22

Discover and visualize data to gain insights■■

Visualize
geographicaldata
Look forcorrelation

Tech Trend Talk: Practical ML 101

2024-07-0915 /22

Discover and visualize data to gain insights■■■

Visualize
geographicaldata
Look forcorrelation
Experiment withattribute
combinations

Tech Trend Talk: Practical ML 101

2024-07-09

15 /22

Project overview

1)Look at the big picture – frame the problem2)Get the data
3)Discover and visualize data to gain insights4)Prepare data for ML algorithms
5)Select a model and train it6)Launch, monitor and maintain

Tech Trend Talk: Practical ML 101

2024-07-09

16 /
22

Prepare data for ML algorithms

Most ML algorithms work best with scalednumerical features

Tech Trend Talk: Practical ML 101

2024-07-09

17 /
22

Prepare data for ML algorithms

Most ML algorithms work best with scaled
numerical features■

Handling
missing data

Tech Trend Talk: Practical ML 101

2024-07-09

17 /
22

Prepare data for ML algorithms

Most ML algorithms work best with scaled
numerical features■■

Handling
missing data
Handling
categoricalfeatures

Tech Trend Talk: Practical ML 101

2024-07-09

17 /
22

Prepare data for ML algorithms

Most ML algorithms work best with scaled
numerical features
Ordinal encoding■■Handlingmissing data
Handling
categoricalfeatures

Tech Trend Talk: Practical ML 101

2024-07-09

17 /22

Prepare data for ML algorithms

Most ML algorithms work best with scaled
numerical features
One-hotencoding■■

Handling
missing data
Handling
categoricalfeatures

Tech Trend Talk: Practical ML 101

2024-07-09

17 /
22

Prepare data for ML algorithms

Most ML algorithms work best with scaled
numerical features■■

Handling missing
data
Handling
categoricalfeatures■

Feature scaling■

Pipeline
Tech Trend Talk: Practical ML 101

2024-07-09

17 /22

Prepare data for ML algorithms

Most ML algorithms work best with scaled
numerical features■■

Handling missing
dataHandlingcategoricalfeatures■Feature scaling■

Pipeline
Tech Trend Talk: Practical ML 101

2024-07-09

17 /22

Project overview

1)Look at the big picture – frame the problem2)Get the data
3)Discover and visualize data to gain insights4)Prepare data for ML algorithms
5)Select a model and train it6)Launch, monitor and maintain

Tech Trend Talk: Practical ML 101

2024-07-09

18 /22

Select a model and train it

Tech Trend Talk: Practical ML 101

2024-07-09

19 /
22

Select a model and train it■

Train a model
Predict

Tech Trend Talk: Practical ML 101

2024-07-09

19 /
22

Select a model and train it■■

Train a model
Predict
Evaluate
performance

Tech Trend Talk: Practical ML 101

2024-07-09

19 /
22

Select a model and train it■■

Train a model
Predict
Evaluate
performance

Tech Trend Talk: Practical ML 101

2024-07-09

19 /
22

Select a model and train it

■

■

■

Train a model
Predict
Evaluate
performance
Avoid overfit withcross-validation

Tech Trend Talk: Practical ML 101

2024-07-09

19 /22

Select a model and train it■■■

Train a model
Predict
Evaluate
performance
Avoid overfit withcross-validation

Tech Trend Talk: Practical ML 101

2024-07-09

19 /22

Select a model and train it

■

■

■

■

Train a model
Predict
Evaluate
performance
Avoid overfit withcross-validation
Try out different
models/parameters

Tech Trend Talk: Practical ML 101

2024-07-09

19 /
22

Select a model and train it

■

■

■

■

Train a model
Predict
Evaluate
performance
Avoid overfit withcross-validation
Try out different
models/parameters

Tech Trend Talk: Practical ML 1012024-07-09

19 /22

Select a model and train it

■

■

■

■

■

Train a model
Predict
Evaluate
performance
Avoid overfit withcross-validation
Try out different
models/parameters
Save with joblib

Tech Trend Talk: Practical ML 101

2024-07-09

19 /22

Project overview

1)Look at the big picture – frame the problem2)Get the data
3)Discover and visualize data to gain insights4)Prepare data for ML algorithms
5)Select a model and train it6)Launch, monitor and maintain

Tech Trend Talk: Practical ML 101

2024-07-09

20 /22

Launch, monitor and maintainLet’s deploy with FastAPI

Tech Trend Talk: Practical ML 101

2024-07-09

21 /
22

Launch, monitor and maintain
Let’s deploy with FastAPI

Tech Trend Talk: Practical ML 101

2024-07-09

21 /
22

Launch, monitor and maintain
Let’s deploy with FastAPI

Tech Trend Talk: Practical ML 101

2024-07-09

21 /
22

Launch, monitor and maintain
Let’s deploy with FastAPI

Tech Trend Talk: Practical ML 101

2024-07-09

21 /
22

Current challenges of ML

Tech Trend Talk: Practical ML 101

2024-07-09

22 /
22

Current challenges of ML■Explainability

Tech Trend Talk: Practical ML 101

2024-07-09

22 /
22

Current challenges of ML■

Explainability■

Ethical and Social Implications

Tech Trend Talk: Practical ML 101

2024-07-09

22 /
22

Current challenges of ML■

Explainability■

Ethical and Social Implications■

Resource Efficiency

Tech Trend Talk: Practical ML 101

2024-07-09

22 /
22

Current challenges of ML■

Explainability■

Ethical and Social Implications■

Resource Efficiency■

Security and Privacy

Tech Trend Talk: Practical ML 101

2024-07-09

22 /
22

Current challenges of ML■

Explainability■

Ethical and Social Implications■

Resource Efficiency■

Security and Privacy■

Lack of “world model”: machines are stupid

Tech Trend Talk: Practical ML 101

2024-07-09

22 /
22

Current challenges of ML■

Explainability■

Ethical and Social Implications■

Resource Efficiency■

Security and Privacy■

Lack of “world model”: machines are stupid■

Multimodal learning–

Type of data: graphs, images, text, ...

–Joint representation and decision making
Tech Trend Talk: Practical ML 101

2024-07-09

22 /22