Reinforcement Learning (REL301m)
Vietnamese title: Học tăng cường
Credits: 3
Prerequisites: AIL303m, DPL302m

Description:
This course helps students understand the concepts of Reinforcement Learning including the space of RL algorithms (Temporal- Difference learning, Monte Carlo, Sarsa, Q-learning, Policy Gradients, Dyna, and more). Students will understand how to formalize these tasks as a Reinforcement Learning problem. Students will learn how RL fits under the broader umbrella of machine learning, and how it complements deep learning, supervised and unsupervised learning. Students can implement a complete RL solution and understand how to apply AI tools to solve real-world problems.

Learning outcomes:
Understand the concepts of Reinforcement Learning
Know how to implement dynamic programming as an efficient solution approach to an industrial control problem
Understand several algorithms for estimating value: Temporal-Difference learning, Monte Carlo
Undersatnd and implement a model-based approach to RL: Temporal-Difference, Expected Sarsa, Q-learning
Understand several approaches that affect approximate value functions: supervised learning, fixed basis, neural network.
Understand how function approximation is used to predict and control in Reinforcement Learning
Understand and implement an RL solution to a problem step by step: problem formulation, appropriate algorithm selection, implementation, and empirical study into the effectiveness of the solution.

Student tasks:
'- Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible to do all exercises, assignments and labs given by instructor in class or at home and submit on time
- Use laptop in class only for learning purpose
- Promptly access to the FU FLM at https://flm.fpt.edu.vn/ for up-to-date course information

Tools:
Jupyter Notebooks, google Colab,

Materials:
description: Fundamentals of Reinforcement Learning-
author: University of Alberta
publisher: Coursera
note: https://www.coursera.org/learn/fundamentals-of-reinforcement-learning
description: Sample-based Learning Methods
author: University of Alberta
publisher: Coursera
note: https://www.coursera.org/learn/sample-based-learning-methods
description: Prediction and Control with Function Approximation
author: University of Alberta
publisher: Coursera
note: https://www.coursera.org/learn/prediction-control-function-approximation
description: A Complete Reinforcement Learning System (Capstone)
author: University of Alberta
publisher: Coursera
note: https://www.coursera.org/learn/complete-reinforcement-learning-system
description: Guided Project
Create a Tic-Tac-Toe game in Python
author: Coursera Project Network
publisher: Coursera
note: https://www.coursera.org/projects/tic-tac-toe-game-python

Assessments:
category: Assignment
type: on-going
weight: 30.0
duration: 2 weeks
question_type: CLO1, CLO2, CLO3, CLO4, CLO5, CLO6, CLO7
no_question: Mini project
knowledge_and_skill: 1
grading_guide: - cover chapter 1, 2, 3
note: at home
category: Lab1
type: on-going
weight: 10.0
duration: 225'
question_type: CLO1, CLO2
no_question: Home work, class
knowledge_and_skill: 1
grading_guide: - cover chapter 1
note: in class, by instructor
category: lab2
type: on-going
weight: 10.0
duration: 225'
question_type: CLO3, CLO4
no_question: Home work, class
knowledge_and_skill: 1
grading_guide: - cover chapter 2
note: in class, by instructor
category: lab3
type: on-going
weight: 10.0
duration: 225'
question_type: CLO5, CLO6
no_question: Home work, class
knowledge_and_skill: 1
grading_guide: - cover chapter 3
note: in class, by instructor
category: Progress test
type: on-going
weight: 10.0
duration: 45'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5, CLO6
no_question: Multiple choices
Marked by Computer or a suitable format
knowledge_and_skill: 30
grading_guide: - cover chapter 1,2,3
note: in class, by instructor
category: Final exam
type: Final exam
weight: 30.0
duration: 60'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5, CLO6, CLO7
no_question: Multiple choices
Marked by Computer
knowledge_and_skill: 50
grading_guide: - cover chapter 1, 2, 3
note: by exam board,
using computer

Schedule:
session: Course Introduction
learning_outcomes: T
materials: mat1
session: 1.1. The K-Armed Bandit Problem
learning_outcomes: T
materials: mat2
session: 1.2. Estimating Action Values
learning_outcomes: T
session: 1.3 Exploration vs. Exploitation Tradeoff
learning_outcomes: T
session: 1.4 Introduction to Markov Decision Processes
learning_outcomes: T
session: 1.5 Goal of Reinforcement Learning
learning_outcomes: T
session: 1.6 Continuing Tasks
learning_outcomes: T
session: 1.7 Policies and Value Functions
learning_outcomes: T
session: 1.8 Bellman Equations
learning_outcomes: T
session: 1.9 Optimality (Optimal Policies & Value Functions)
learning_outcomes: T
session: 1.10 Policy Evaluation (Prediction)
learning_outcomes: T
session: 1.11 Policy Iteration (Control)
learning_outcomes: T
session: 1.12 Generalized Policy Iteration
learning_outcomes: T
session: Lab 1
learning_outcomes: U
session: Lab 1
learning_outcomes: U
session: Lab 1
learning_outcomes: U
session: Lab 1
learning_outcomes: U
session: 4.1 Assignment introduction
session: 2.1 Introduction to Monte-Carlo Methods
learning_outcomes: T
session: 2.2 Monte-Carlo for Control
learning_outcomes: T
session: 2.3 Exploration Methods for Monte-Carlo
learning_outcomes: T
session: 2.4 Off-policy learning for prediction
learning_outcomes: T
session: 2.5 Introduction to Temporal Difference Learning
learning_outcomes: T
session: 2.6 Advantages of Temporal Difference
learning_outcomes: T
session: 2.7 Temporal Difference for Control
learning_outcomes: T
session: 2.8 Off-policy Temporal Difference Control: Q-learning
learning_outcomes: T
session: 2.9 Expected Sarsa
learning_outcomes: T
session: 2.10 Define model in Reinforcement Learning
learning_outcomes: T
session: 2.11 Define Planning in Reinforcement Learning
learning_outcomes: T
session: 2.12 Dyna as a formalism for planning
learning_outcomes: T
session: 2.13 Dealing with inaccurate models
learning_outcomes: T
session: Lab 2
learning_outcomes: U
session: Lab 2
learning_outcomes: U
session: Lab 2
learning_outcomes: U
session: Lab 2
learning_outcomes: U
session: 4.2 Assignment guide 1
session: 3.1 Estimating Value Functions as Supervised Learning
learning_outcomes: T
session: 3.2 The Objective for On-policy Prediction
learning_outcomes: T
session: 3.3 The Objective for Temporal Difference
learning_outcomes: T
session: 3.4 Linear Temporal Difference
learning_outcomes: T
session: 3.5 Feature Construction for Linear Methods
learning_outcomes: T
session: 3.6 Episodic Sarsa with Function Approximation
learning_outcomes: T
session: 3.7 Exploration under Function Approximation
learning_outcomes: T
session: 3.8 Understand Average Reward
learning_outcomes: T
session: 3.9 Learning Parameterized Policies
learning_outcomes: T
session: 3.10 Policy Gradient for Continuing Tasks
learning_outcomes: T
session: 3.11 Actor-Critic for Continuing Tasks
learning_outcomes: T
session: 3.12 Policy Parameterizations
learning_outcomes: T
session: Lab 3
learning_outcomes: U
session: Lab 3
learning_outcomes: U
session: Lab 3
learning_outcomes: U
session: Lab 3
learning_outcomes: U
session: 4.3 Assignment guide 2
session: Guided Project: Create a Tic-Tac-Toe game in Python
learning_outcomes: U
session: Guided Project: Create a Tic-Tac-Toe game in Python
learning_outcomes: U
session: Quiz
learning_outcomes: T
session: Assignment- Assignment grading
learning_outcomes: U
session: Assignment- Assignment grading
learning_outcomes: U
session: Assignment- Assignment grading
learning_outcomes: U
session: Review course
Results
learning_outcomes: U