Nagoya University, Fall 2023

Lecture 1

Course information & Overview,
Google colab & Python,
Supervised learning I
https://www.henrikbachmann.com/mml2023.html

Who is teaching & helping?
Lecturer

Henrik Bachmannwww.henrikbachmann.com
henrik.bachmann@math.nagoya-u.ac.jp
Teaching AssistantRisan

www.risan.io

NUEMI Tutors
Caleb
PrathamHeviduMuflihRafi

Zhanfeng
RaymondAi
AlbertoWhat where?
Homepagehttps://www.henrikbachmann.com/mml2023.html
Contains schedule & course materials

Discord server•

https://discord.gg/XG2qkFyGpD

Will be used for discussions and small announcements

TACT

https://tact.ac.thers.ac.jp/portal/site/n_2023_0061611

Will be used for Homework submission & importantannouncements

For who is this lecture?
•G30 Students of year >= 2.
(For first year students: Consider taking this course in your second year)••

NUPACE Students.
Japanese students who want to attend English lectures.

Basic knowledge in Linear Algebra and Calculus is helpful(and sometimes necessary to understand everything in detail)If you are Japanese student, please register at the NUEMI project for this lecture:
http://labguide.bio.nagoya-u.ac.jp/NUEMI/registration/

Free extra support & Study sessions

What is this course?
• This course is a “Special Mathematics Lecture”(Introduced by Prof. Richard)• Can give 2 credit points depending on your year.
• For some students can count for GPA.
• You need to be registered in the NU Portal if you wantthese credits.
• Details: http://www.math.nagoya-u.ac.jp/~richard/Files/SML_official_rule.pdf
Goal of this lecture:•••

Get an overview of some algorithms used in machine learning
Understand the mathematics used in these algorithms
Learn how to implement (simple version of) them in Python

Not goal of this lecture: Give detailed overview of machine learning libraries(tensorflow etc.) and learn how to use themGrading
Grading will be based on• Homework assignments (70%)• Semester project (30%)
Homework assignments••••

Programming exercises in Google ColabIn total 5 Homework assignments
14 points each Homework
Submission in TACT

Semester project
Objective: Choose a project topic related to Nagoya or Japan more broadly thatcan be addressed using machine learning algorithms. Your task is to develop amachine learning model to solve a specific problem or provide insights into an
aspect of life, business, environment, culture, etc., in Nagoya/Japan.
Group Size: 1-3 members
Code: Preferably a Google Colab notebook. Exceptions are possible; pleaseprovide full documentation for any different technology or package used. If you
plan not to submit a Google Colab, please contact us in advance.
Documentation: 5-10 slides as if you were going to present the project.
Your slides should cover (for example)::odott• Problem Statementahwlpeeulhc/• Data Collection
oasneediv
rehaouks• Data Exploration and Visualization
moeyeefv
Igigw
lnliiwm• Model Building and Evaluation
coWeehtg• Conclusion
durin

Tutorial
Time: Tuesday? Thursday? Friday? 6th period(vote now via menti.com)
Place: Same room (A250, Science Bld. A)• Organized by the TAs.
• Talk again about the content of the lecture.
• Do the Assignments with the help of TAs &other students.

What is machine learning
Common definition:
Machine Learning is the study of computer algorithms
that improve automatically through experience
Found its way in a lot of daily life applications, e.g.
• Self driving/flying cars/trains/planes/rockets• Computer games• Image & Video & Sound processing• Speech recognition (Siri, Alexa,…)• Translation (Google translate)• Weather forecast• Data analysis• Email spam
• Robotics• ChatGPT• and much much more…

Not machine learning:“Classical machines / programs”
Programmer understands theproblem and can implement an
explicit algorithm, which turns an
input into an output.

Example: Determinant of a matrix
detWhat do you see?

Want: Cat & Dog detection machine
C&D“This is a dog!”
Cat & Dog detector

Of course, I know how cats and dogs
look like. But I do not know how todescribe it with a formula or algorithm!
Smart person

How do we know?
This is a dog because…
•I saw a lot of dogs & cats in my life.

•Whenever I saw a cat I was told by my parents„This is a cat!“. When I saw a dog they told me“This is a dog.”.

•I learned to distinguish between dogs and cats.

•But it is hard for me to explain the exact
difference.

Machine learning
We can create machines/programs which we can teachCatDog

After Training

Training

Dog

Dog!
… with 95% certainty!

Cat & Dog detector

Machine learning overview

Supervised learning
Classification (discrete output)

Output = category (e.g. “dog”, “cat” or “1”,”2”,…,”9”)”2”
Handwriting recognition

Regression (continuous output)

Output = real number (like “price”, “weight”,…)
Size: 70#!
Rooms: 3

Learning by using some trainingsdata90.000 Yen
Apartment price prediction

Unsupervised learning

Clustering

Dimension reduction

Reinforcement learning

Examples of AI in Chess, Go, Computer games, biology, math••••••1996: Deep blue (not machine learning)
2016: Alpha Go
2017: Alpha Zero (plays chess, shōgi & go)ningrael
2019: AlphaStar (Starcraft 2)
t
en
mecro
2020: AlphaFold
Reinf
2022: ChatGPT, AlphaTensor
1996: Garry Kasparov (Chess worldchampion) loses against Deep blue.

2016: Lee Sedol loses against Alpha Go.

2019: AlphaStar beats Starcraft 2 pro gamers

2022: ChatGPT… for lab reports and moreOver of the topics in this course
This year:
1. Introduction & Python
2. Supervised learning: Linear regression
3. Supervised learning: Neural networks (from scratch & advanced topics)
4. Unsupervised learning: Dimension reduction (e.g., PCA, Autoencoder)
5. Reinforcement learning: (deep) Q-learning
6. Language models (if time allows)

Fall 2022
1.
2.
3.
4.
5.
6.

Introduction to programming in Python
Supervised learning: Linear & logistic regression
Generative Learning algorithms: Naive Bayes
Reinforcement Learning: Q-LearningUnsupervised learning: k-means clustering
Neural networks

Let us have a look at Google Colab andHomework 1

1 Supervised learning: Tebasaki example
Have: Some data of ”Weeks living in Nagoya” and “Tebasaki eaten”.
Want: A functions, which creates out of an an arbitrary input for “Weeks livingin Nagoya” a prediction for “Tebasaki eaten”.

Yamachan & Furaibo – possible places toincrease the number of ”Tebasaki eaten”

learning Some notations
1 Supervised learning:
Training set

featurelabel (target)

Training example

Learning algorithm(arbitrary number of)
Weeks living in Nagoya(feature)

Predicted number ofeaten Tebasaki

hypothesis(label)1 Supervised learning: Notations

Tebaraki Example
AGFEX

2,9 7 RdRxRTµR N27138

Tebasaki example:

Idea

Fotohohl
OF ER

Tebasaki example:

1 Supervised learning – Linear Regression
Learning Algorithm: Linear Regression

EaTebasaki example

d1X

x

Goal: Determine the “best “parameters for a given trainings set.

1 Supervised learning – Linear Regression
Measure how good parameters are:Goal rephrased: Minimize the cost function for a given trainings set.

There are several different choices for cost functions. The above choicecorresponds is the “least-squares cost function“.

1 Supervised learning – How to minimize a function?

FAfdIRhinRIn

whereasFi

2

Calculate f xd

1 Supervised learning – Linear Regression
Gradient descent main idea:

Fact: The gradient shows in the direction of the steepest ascent