Deep Learning (DPL302m)
Vietnamese title: Học sâu
Credits: 6
Prerequisites: AIL303m

Description:
In this course, student will build and train neural network architectures such as Convolutional Neural Networks, Recurrent Neural Networks, LSTMs, Transformers, and learn how to make them better with strategies such as Dropout, BatchNorm, Xavier/He initialization, and more. Get ready to master theoretical concepts and their industry applications using Python and TensorFlow and tackle real-world cases such as speech recognition, music synthesis, chatbots, machine translation, natural language processing, and more.

Learning outcomes:
Should build, train and apply fully connected deep neural networks, implement efficient neural networks, identify key parameters in a neural network's architecture, and apply deep learning to students' own applications.
Could train and develop test sets, analyze bias/variance for building deep learning applications, apply optimization algorithms, and implement a neural network using TensorFlow.
Understand complex ML settings, diagnose errors in a machine learning system, prioritize strategies for reducing errors, and apply end-to-end learning, transfer learning, and multi-task learning.
Build CNN models for visual detection and recognition tasks and use neural style transfer to generate art from various types of data.
Could build and train recurrent neural networks (RNNs), work with NLP and word embeddings, use HuggingFace tokenizers, and transformer models to solve different NLP tasks.
Implement a DL project includes steps: data collection, data wrangling, exploratory data analysis, model development, model evaluation, and reporting
Understand how to apply AI tools to complete a mini project

Student tasks:
- Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible to do all exercises, assignments and labs given by instructor in class or at home and submit on time
- Use laptop in class only for learning purpose
- Promptly access to the FU LMS at http://flm.fpt.edu.vn for up-to-date course information

Tools:
- Jupyter notebook
- Cloud server: 72 hours/project

Materials:
description: Convolutional Neural Networks
author: DeepLearning.AI
publisher: Coursera
note: https://www.coursera.org/learn/convolutional-neural-networks?specialization=deep-learning
description: Deep Learning Specialization
author: DeepLearning.AI
publisher: Coursera
note: https://www.coursera.org/specializations/deep-learning
description: GP 1: Diabetes Disease Detection with XG-Boost and Neural Networks
publisher: Coursera
note: https://www.coursera.org/projects/diabetes-disease-detection-with-xg-boost-and-neural-networks
description: GP 2: Hyperparameter Tuning with Keras Tuner
publisher: Coursera
note: https://www.coursera.org/projects/keras-tuner
description: GP 3: Object Localization with TensorFlow
publisher: Coursera
note: https://www.coursera.org/projects/object-localization-tensorflow
description: GP 4: NLP: Twitter Sentiment Analysis
publisher: Coursera
note: https://www.coursera.org/projects/twitter-sentiment-analysis
description: Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization
author: DeepLearning.AI
publisher: Coursera
note: https://www.coursera.org/learn/deep-neural-network?specialization=deep-learning
description: Neural Networks and Deep Learning
author: DeepLearning.AI
publisher: Coursera
note: https://www.coursera.org/learn/neural-networks-deep-learning?specialization=deep-learning
description: Sequence Models
author: DeepLearning.AI
publisher: Coursera
note: https://www.coursera.org/learn/nlp-sequence-models?specialization=deep-learning
description: Structuring Machine Learning Projects
author: DeepLearning.AI
publisher: Coursera
note: https://www.coursera.org/learn/machine-learning-projects?specialization=deep-learning

Assessments:
category: Progress test
type: on-going
weight: 10.0
duration: 60'
question_type: CLO1, CLO2, CLO3, CLO4
no_question: Multiple choices Marked by Computer or a suitable format
knowledge_and_skill: 50
grading_guide: - cover chapter 1,2,3
note: in class, by instructor
category: Report 1: Project Proposal [ Oral group presentation]
type: on-going
weight: 10.0
duration: 225'
question_type: CLO6, CLO7
no_question: Presentation, class
knowledge_and_skill: 1
grading_guide: - cover chapter 1
note: in class, by instructor
category: Report 2: Data Tasks [ Oral group presentation]
type: on-going
weight: 10.0
duration: 225'
question_type: CLO6, CLO7
no_question: Presentation, class
knowledge_and_skill: 1
grading_guide: - cover chapter 2
note: in class, by instructor
category: Report 3: Model & Results [ Oral group presentation]
type: on-going
weight: 30.0
duration: 225'
question_type: CLO6, CLO7
no_question: Presentation, class
knowledge_and_skill: 1
grading_guide: - cover chapter 3
note: in class, by instructor
category: Report 4: Final report & Group presentation
type: on-going
weight: 10.0
duration: 225'
question_type: CLO6, CLO7
no_question: Presentation, class
knowledge_and_skill: 1
grading_guide: - cover chapter 4
note: in class, by instructor
category: Final exam
type: Final exam
weight: 30.0
duration: 60'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5
no_question: Multiple choices
Marked by Computer
knowledge_and_skill: 50
grading_guide: - cover chapter 1, 2, 3, 4
note: by exam board,
using computer

Schedule:
session: Course Introduction
1.1. Introduction to deep learning
learning_outcomes: IT
materials: mat1
session: 1.2 Neural Networks Basics
learning_outcomes: T
materials: mat2
session: 1.2 Neural Networks Basics
learning_outcomes: T
materials: mat3
session: 1.2 Neural Networks Basics
learning_outcomes: T
materials: mat4
session: 1.3 Shallow neural networks
learning_outcomes: T
session: 1.3 Shallow neural networks
learning_outcomes: T
session: 1.3 Shallow neural networks
learning_outcomes: T
session: Discussion on the Project (CP) topic
learning_outcomes: TU
session: Formation of teams/groups
learning_outcomes: TU
session: Discuss and choose a project topic
learning_outcomes: TU
session: Report 1: Project Proposal (Discussion)
learning_outcomes: U
session: Report 1: Project Proposal (Discussion)
learning_outcomes: U
session: 1.4 Deep Neural Networks
learning_outcomes: T
session: 1.4 Deep Neural Networks
learning_outcomes: T
session: 1.4 Deep Neural Networks
learning_outcomes: T
session: Team Meeting: Project Proposal Writing Report 1)
learning_outcomes: U
session: Team Meeting: Project Proposal Writing (Report 1)
learning_outcomes: U
session: Team Meeting: Project Proposal Writing (Report 1)
learning_outcomes: U
session: 1.4 Deep Neural Networks
learning_outcomes: T
session: 1.4 Deep Neural Networks
learning_outcomes: T
session: 1.4 Deep Neural Networks
learning_outcomes: T
session: Team Meeting: Instructor's feedbacks for Project Proposal (Report 1)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Project Proposal (Report 1)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Project Proposal (Report 1)
learning_outcomes: TU
session: 2.1 Practical aspects of Deep Learning
learning_outcomes: T
session: 2.1 Practical aspects of Deep Learning
learning_outcomes: T
session: 2.1 Practical aspects of Deep Learning
learning_outcomes: T
session: Team Meeting: Data Collection (Present the issues discussed and agreed upon by the group) (Report 2)
learning_outcomes: U
session: Team Meeting: Data Collection (Present the issues discussed and agreed upon by the group) (Report 2)
learning_outcomes: U
session: Team Meeting: Data Collection (Present the issues discussed and agreed upon by the group) (Report 2)
learning_outcomes: U
session: Team Meeting: Data Collection (Present the issues discussed and agreed upon by the group) (Report 2)
learning_outcomes: U
session: Team Meeting: Data Collection (Present the issues discussed and agreed upon by the group) (Report 2)
learning_outcomes: U
session: Team Meeting: Data Collection (Present the issues discussed and agreed upon by the group) (Report 2)
learning_outcomes: U
session: 2.2 Optimization algorithms
learning_outcomes: T
session: 2.2 Optimization algorithms
learning_outcomes: T
session: 2.2 Optimization algorithms
learning_outcomes: T
session: Team Meeting: Instructor's feedbacks for Data Tasks all phase (Report 2)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Data Tasks all phase (Report 2)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Data Tasks all phase (Report 2)
learning_outcomes: TU
session: 2.3 Hyperparameter tuning, Batch Normalization and Programming Frameworks
learning_outcomes: T
session: 2.3 Hyperparameter tuning, Batch Normalization and Programming Frameworks
learning_outcomes: T
session: 2.3 Hyperparameter tuning, Batch Normalization and Programming Frameworks
learning_outcomes: T
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: 2.3 Hyperparameter tuning, Batch Normalization and Programming Frameworks
learning_outcomes: T
session: 3.1 ML Strategy 1
learning_outcomes: T
session: 3.1 ML Strategy 1
learning_outcomes: T
session: 3.1 ML Strategy 2
learning_outcomes: T
session: 3.2 ML Strategy 2
learning_outcomes: T
session: 3.2 ML Strategy 2
learning_outcomes: T
session: 4.1 Foundations of CNNs
learning_outcomes: T
session: 4.1 Foundations of CNNs
learning_outcomes: T
session: 4.1 Foundations of CNNs
learning_outcomes: T
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: 4.2 Deep convolutional models: case studies
learning_outcomes: T
session: 4.2 Deep convolutional models: case studies
learning_outcomes: T
session: 4.2 Deep convolutional models: case studies
learning_outcomes: T
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: 4.2 Deep convolutional models: case studies
learning_outcomes: T
session: 4.2 Deep convolutional models: case studies
learning_outcomes: T
session: 4.2 Deep convolutional models: case studies
learning_outcomes: T
session: 4.3 Object detection
learning_outcomes: T
session: 4.3 Object detection
learning_outcomes: T
session: 4.3 Object detection
learning_outcomes: T
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: 4.3 Object detection
learning_outcomes: T
session: 4.3 Object detection
learning_outcomes: T
session: 4.3 Object detection
learning_outcomes: T
session: 4.3 Object detection
learning_outcomes: T
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: 4.4 Special applications: Face recognition and Neural style transfer
learning_outcomes: T
session: 4.4 Special applications: Face recognition and Neural style transfer
learning_outcomes: T
session: 4.4 Special applications: Face recognition and Neural style transfer
learning_outcomes: T
session: 4.4 Special applications: Face recognition and Neural style transfer
learning_outcomes: T
session: 5.1 Recurrent Neural Networks
learning_outcomes: T
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: Team Meeting: Model & Results (Present the issues discussed and agreed upon by the group) (Report 3)
learning_outcomes: U
session: 5.1 Recurrent Neural Networks
learning_outcomes: T
session: 5.1 Recurrent Neural Networks
learning_outcomes: T
session: 5.1 Recurrent Neural Networks
learning_outcomes: T
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Model & Results (Report 3)
learning_outcomes: TU
session: Progess test
learning_outcomes: T
session: 5.2 Natural Language Processing and Word Embeddings
learning_outcomes: T
session: 5.2 Natural Language Processing and Word Embeddings
learning_outcomes: T
session: 5.3 Sequence models and Attention mechanism
learning_outcomes: T
session: Team Meeting: Final Report (Present the issues discussed and agreed upon by the group)
learning_outcomes: U
session: Team Meeting: Final Report (Present the issues discussed and agreed upon by the group)
learning_outcomes: U
session: Team Meeting: Final Report (Present the issues discussed and agreed upon by the group)
learning_outcomes: U
session: 5.4 Transformer Network
learning_outcomes: T
session: 5.4 Transformer Network
learning_outcomes: T
session: 5.4 Transformer Network
learning_outcomes: T
session: 5.4 Transformer Network
learning_outcomes: T
session: Team Meeting: Instructor's feedbacks for Final Report (Report 4)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Final Report (Report 4)
learning_outcomes: TU
session: Team Meeting: Instructor's feedbacks for Final Report (Report 4)
learning_outcomes: TU
session: Project defense
learning_outcomes: TU
session: Project defense
learning_outcomes: TU
session: Project defense
learning_outcomes: TU
session: Project defense
learning_outcomes: TU
session: Project defense
learning_outcomes: TU
session: Project defense
learning_outcomes: TU
session: Review
learning_outcomes: T