Mathematics for Machine Learning (MAI391)
Vietnamese title: Toán cho học máy
Credits: 3
Prerequisites: MAE101

Description:
This course introduces the mathematical concepts and foundations needed to talk about the three main components of a machine learning system: data, models, and learning. Upon this course, students will be able to understand: • Foundametal concepts about matrices and matrix decomposition.
• Concepts of gradients.
• The basics of probability and some distributions.
• Optimization to find maxima/minima of functions. • Dimensionality reduction using principal component analysis.
• Classification in the context of support vector machines.
Method of teaching and learning: Lecture and project based learning

Learning outcomes:
Analytic Geometry LO 0.1 Define a norm and an inner product on a vector space LO 0.2 Determine bilinear mapping and check if a square matrix is positive definite. LO 0.3 Find the length of a vector as well as the distance between two vectors in a vector space by using inner product. LO 0.4 Find the angle between two vectors and check if they are orthogonal. Check if a square matrix is orthogonal. LO 0.5 Check if a set of vectors is an orthogonal or an orthonormal basis of a given vector space. Determine an orthogonal complement of a subspace. LO 0.6 Construct the projection onto a subspace.
Matrix Decomposition LO 1.1 Compute the determinant of a square matrix, decide when a square matrix is invertible by equivalent conditions. Find the characteristic polynomial, eigenvalues and eigenvectors of a matrix. Understand some theorems concerning eigenvalues. LO1.2 Diagonalize a matrix by eigendecomposition. Decompose a matrix by singular value decomposition. LO1.3 Compute matrix approximation.
Vector Calculus LO 2.1 Revisit differentiation of a univariate function. Find Taylor expansion of a smooth function at a given point. LO 2.2 Find partial derivatives using basic formulas and the chain rule, gradients of multivariate functions. LO 2.3 Find gradients, Jacobian matrices of Vector-Valued Functions. LO 2.4 Take gradients of matrices with respect to vectors. LO 2.5 Understand the idea of backpropagation in deep learning. LO 2.6 Find the higher-order partial derivatives, linearization and multivariate Taylor series of multivariate functions.
Probability and Distributions LO 3.1 Catch the notions of discrete variable continuous variable; find the probability mass function, the probability density function, the cumulative distribution function and their properties. LO 3.2 Calculate conditional probabilities; use multiplication rule, total probability rule and Bayes theorem to find probabilities. LO 3.3 Compute the mean and variance of a random variable, compute the covariance and the correlation between two variables. LO 3.4 Consider whether two variables are independent. LO 3.5 Describe a Gauss distribution, marginals, conditionals, sums and products of Gauss distributions.
Continuous Optimization LO 4.1 Find a local minimum of a function using gradient descent. LO 4.2 Use Lagrange multipliers to find the maximum/minimum value of a multivariate function with constraints. LO 4.3 Understand convex optimization problems and solve linear programings.
Linear Regression LO 5.1 Use maximum likelihood and maximum a posteriori (MAP) estimation to find optimal model parameters. LO 5.2 Using these parameter estimates to have a brief look at generalization errors and overfitting. LO 5.3 Discuss Bayesian linear regression.
Dimensionality Reduction with PCA LO 6.1 Compute the data covariance matrix of a data set, construct m-dimensional subspace with maximum variance. LO 6.2 Find the basis of theprincipal subspace. LO 6.3 Using Low-Rank Matrix to approximate PCA. LO 6.4 Discuss PCA in high dimension and key steps of PCA.
Classification with SVM LO 7.1 Understand binary classification by seperating hyperplane. LO 7.2 Catch mathematical ideas of support vector machine. LO 7.3 Understand the advantage of dual support vector machine. LO 7.4 Introduce non-linear classifiers by kernels. LO 7.5 Consider two different approaches for finding the optimal solution for the SVM.
Attitude and soft skills LO 8 Develop working in group skills and self-discipline

Student tasks:
- Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible for doing all exercises and assignments given by the instructor in class or at home.
- Use a laptop in class only for learning purposes
- Promptly access course websites for up-to-date course information

Tools:
- Internet
- Python Programme

Materials:
description: Mathematics for Machine Learning
author: M. P. Deisenroth, A. Faisal, Ch. S. Ong
publisher: Cambrige Univeristy Press
published_date: 2020
edition: 1st
note: https://mml-book.github.io/?fbclid=IwAR2WAfBUZe-E6O-pbbGL3k6E7KngL8F9sbOqraZr9LD14PAnHw6Y_e-O6Q4
description: PDF Slides
author: Instructors
description: Machine Learning cơ bản
author: Vũ Hữu Tiệp
note: https://github.com/tiepvupsu/ebookMLCB/blob/master/book_ML.pdf
description: Multivariate Calculus
author: Denis Auroux
publisher: MIT OpenCourseWare
note: https://ocw.mit.edu/courses/mathematics/18-02sc-multivariable-calculus-fall-2010/index.htm
description: Mathematics for machine learning
author: Mirror Neuron Chanel
note: https://www.youtube.com/playlist?list=PLcQCwsZDEzFmlSc6levE3UV9rZ8yY-D_7
description: Mathematics for machine Learning
author: Digital Learning Hub - Imperial College London Chanel
note: https://www.youtube.com/channel/UCSzae1ITUdw9DCdELMduaQw/playlists
description: Linear Algebra Done Right
author: Sheldon Axler
publisher: Springer
published_date: 2015
edition: 3rd
description: Machine Learning
author: Andrew Ng
publisher: Stanford Chanel
note: https://www.youtube.com/watch?v=UzxYlbK2c7E&list=PLA89DCFA6ADACE599&index=1
description: Mathematics for Machine Learning
author: Cousera
note: https://www.coursera.org/specializations/mathematics-machine-learning

Assessments:
category: Assignments
type: on-going
weight: 30.0
duration: 3 hours
question_type: LO0, LO1, LO2, LO3, LO4, LO5, LO8
no_question: open
grading_guide: OP1: Assigment 1: Cover chapters 3, 4,5 Assignment 2: Cover chapters 6,7 Assignment 3: Cover chapters 9, 10, 12 OP2: (For Constructivism Approach only): Follow lecturer's proposal
note: Each assignment is divided into two parts: the take-home component counts for no more than 50% of the total grade, and the in-class component counts for at least 50% (this may take the form of a presentation, essay, problem-solving, etc.).
category: Computer Project
type: on-going
weight: 20.0
duration: 5 hours
question_type: LO5, LO6, LO7, LO8
no_question: OP1: open // OP2: (For Constructivism Approach only): Follow lecturer's proposal
grading_guide: cover chapter 9, 10,12
note: Students are divided into groups of 3-5 students. Each group will choose a Machine Learning algorithm (with instructor’s consent) and write code to implement the algorithm. Each group is required to explain the mathematic background of the chosen algorithm, run the model on a real data and present their work to the class. The work (slides, source files, etc.) accounts for up to 50% of the grade, while presentation skills and the ability to answer questions account for at least 50%.
category: Progress tests
type: on-going
weight: 20.0
duration: 30'
question_type: LO0, LO1, LO2, LO3, LO4
no_question: OP1: Multiple choices // OP2: (For Constructivism Approach only): Follow lecturer's proposal
knowledge_and_skill: 12
grading_guide: - Test 1: cover chapters 3, 4,5 - Test 2: cover chapters 6,7
note: by instructor, and by
suitable means (computer, paper, oral)
category: Final exam
type: Final exam
weight: 30.0
duration: 60'
question_type: LO 0, LO 1, LO 2, LO 3, LO 4
no_question: Multiple choices
Marked by Computer
knowledge_and_skill: 40
grading_guide: Selected chapters
note: by exam board,
using computer

Schedule:
session: Chapter 3. Analytic Geometry
3.1 Norm
3.2 Inner Products
learning_outcomes: T
U
session: 3.3 Lengths and Distances
3.4 Angles and Orthogonality
learning_outcomes: T
U
session: 3.5 Orthonormal Basis
3.6 Orthonormal Complement
learning_outcomes: T
U
session: 3.8 Orthogonal Projections
learning_outcomes: T
U
session: Chapter 4. Matrix Decompositions
4.1 Determinants and Traces
learning_outcomes: T
U
session: 4.2 Eigenvalues and Eigenvectors
learning_outcomes: T
U
session: 4.4 Eigendecomposition and Diagonalization
learning_outcomes: T
U
session: 4.5 Singular value Decomposition
learning_outcomes: T
U
session: 4.5 Singular value Decomposition (cont.)
learning_outcomes: T
session: 4.6 Matrix Approximation
learning_outcomes: T
session: Chapter 5. Vector Calculus
5.1 Differentiation of Univariate Functions
learning_outcomes: T
session: 5.2 Partial Differentiation and Gradients
learning_outcomes: T
U
session: 5.2 Partial Differentiation and Gradients (cont.)
learning_outcomes: T
U
session: 5.3 Gradients of Vector-Valued Functions
learning_outcomes: T
U
session: 5.3 Gradients of Vector-Valued Functions (cont.)
learning_outcomes: T
U
session: 5.6 Backpropagation and Automatic Differentiation
learning_outcomes: T
session: 5.7 Higher-Order Derivatives
learning_outcomes: T
U
session: 5.8 Linearization and Multivariate Taylor Series
learning_outcomes: T
U
session: Review
learning_outcomes: U
session: Progress Test 1
Submission of Assigment 1
session: Chapter 6. Probability and Distributions
6.1 Construction of a Probability Space
learning_outcomes: T
session: 6.2 Discrete and Continuous Probabilities
learning_outcomes: T
U
session: 6.3 Sum Rule, Product Rule, and Bayes’ Theorem
learning_outcomes: T
U
session: 6.3 Sum Rule, Product Rule, and Bayes’ Theorem (cont.)
learning_outcomes: T
U
session: 6.4 Summary Statistics and Independence
learning_outcomes: T
U
session: 6.4 Summary Statistics and Independence (cont.)
learning_outcomes: T
U
session: 6.5 Gaussian Distribution
learning_outcomes: T
session: 6.5 Gaussian Distribution (cont.)
learning_outcomes: T
U
session: Chapter 7. Continuous Optimization
7.1 Optimization Using Gradient Descent
learning_outcomes: T
U
session: 7.2 Constrained Optimization and Lagrange Multipliers
learning_outcomes: T
session: 7.2 Constrained Optimization and Lagrange Multipliers (cont.)
learning_outcomes: T
session: 7.3 Convex Optimization
learning_outcomes: T
session: 7.3 Convex Optimization (cont.)
learning_outcomes: T
U
session: 7.3 Convex Optimization (cont.)
learning_outcomes: T
U
session: Review
learning_outcomes: U
session: Progress Test 2
Submission of Assignment 2
session: Chapter 9. Linear Regression
9.1 Problem Formulation
learning_outcomes: T
session: 9.1 Problem Formulation (cont.)
learning_outcomes: T
session: 9.2 Parameter Estimation
learning_outcomes: T
session: 9.2 Parameter Estimation (cont.)
learning_outcomes: T
U
session: Optional:
9.3 Bayesian Linear Regression
learning_outcomes: T
session: Optional:
9.4 Maximum Likelihood as Orthogonal Projection
learning_outcomes: T
session: Chapter 10. Dimensionality Reduction with Principal Component Analysis
10.1 Problem Setting
learning_outcomes: T
session: 10.2 Maximum Variance Perspective
learning_outcomes: T
session: 10.2 Maximum Variance Perspective (cont.)
learning_outcomes: T
U
session: 10.6 Key Steps of PCA in Practice
learning_outcomes: T
session: 10.6 Key Steps of PCA in Practice (cont.)
learning_outcomes: T
U
session: Optional:
10.3 Projection Perspective
learning_outcomes: T
session: Optional:
10.4 Eigenvector Computation and Low-Rank Approximations
learning_outcomes: T
U
session: Optional:
10.5 PCA in High Dimensions
learning_outcomes: T
U
session: Chapter 12. Classification with Support Vector Machines
12.1 Separating Hyperplanes
learning_outcomes: T
session: 12.2 Primal Support Vector Machine
learning_outcomes: T
session: 12.2 Primal Support Vector Machine (cont.)
learning_outcomes: T
U
session: 12.2 Primal Support Vector Machine (cont.)
learning_outcomes: T
session: 12.3 Dual Support Vector Machine
learning_outcomes: T
session: 12.3 Dual Support Vector Machine (cont.)
learning_outcomes: T
U
session: Optional:
12.4 Kernels
learning_outcomes: T
session: Optional:
12.5 Numerical Solution
learning_outcomes: T
session: Presentation of Computer Project
learning_outcomes: U
session: Presentation of Computer Project (cont.)
learning_outcomes: U