Speech Processing (SLP301)
Vietnamese title: Xử lý tiếng nói
Credits: 3
Prerequisites: CSD203, AIL303m

Description:
This speech processing course comprehensively explains how humans and machines generate, transmit, and process speech signals. The course covers speech production, phonetics, acoustic analysis, speech perception, speech recognition, natural language processing, and speech synthesis. Students will learn about the principles and algorithms behind various speech-processing techniques, including signal processing, machine learning, and deep learning. The course also covers speech processing applications in speech recognition, speech recognition, speech synthesis, and speech enhancement. Upon completing the course, students will have a solid foundation in speech processing and be equipped with the necessary skills to tackle real-world speech-processing problems.

Learning outcomes:
Gain a comprehensive understanding of fundamental principles in processing speech signals, covering digital signal processing (DSP) methods, feature extraction, and speech signal analysis
Develop expertise in speech recognition and synthesis, exploring and applying various algorithms for automatic speech recognition (ASR) and speech synthesis, including rule-based and statistical approaches
Apply relevant machine learning algorithms and models to address speech-processing challenges, including feature extraction, classification, and other tasks within the speech context
Acquire practical skills by using and implementing widely-used speech-processing tools, software, libraries, and frameworks, such as Praat, HTK, or other relevant platforms
Demonstrate the ability to assess speech-processing system performance, applying appropriate metrics to evaluate accuracy and efficiency in speech recognition and synthesis models. Critically analyze the strengths and limitations of various approaches

Student tasks:
'- Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible to do all exercises, assignments and labs given by instructor in class or at home and submit on time
- Use laptop in class only for learning purpose
- Promptly access to the FU FLM at http://flm.fpt.edu.vn for up-to-date course information

Tools:
No

Materials:
description: Natural Language Processing and Computational Linguistics 1: Speech, Morphology and Syntax
author: Mohamed Zakaria Kurdi
publisher: Wiley
published_date: 2016
isbn: 978-1848218482 (ISBN-13)
note: https://library.books24x7.com
description: Spoken Language Processing
author: Joseph Mariani
publisher: Wiley
published_date: 2009
isbn: 978-1848210318 (ISBN-13)
note: https://library.books24x7.com
description: Theory and Applications of Digital Speech Processing
author: Lawrence Rabiner, Ronald Schafer
publisher: Pearson
published_date: 2010
isbn: 978-0136034285 (ISBN-13)
description: Speech and Language Processing
author: Daniel Jurafsky, James Martin
publisher: Prentice Hall
published_date: 2008
isbn: 978-0131873216 (ISBN-13)

Assessments:
category: Assignment 1
type: on-going
weight: 15.0
duration: 45'
question_type: CLO1
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content CLO1
note: in class, by group presentation
category: Assignment 2
type: on-going
weight: 15.0
duration: 45'
question_type: CLO2,CLO3
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content CLO2,CLO3
note: in class, by group presentation
category: Progress test 1
type: on-going
weight: 15.0
duration: 20'-30'
question_type: CLO1,CLO2
no_question: Multiple choicesMarked by Computer or a suitable format
knowledge_and_skill: 30
grading_guide: - cover content CLO1,CLO2
note: in class, by instructor
category: Progress test 2
type: on-going
weight: 15.0
duration: 20'-30'
question_type: CLO3,CLO4,CLO5
no_question: Multiple choicesMarked by Computer or a suitable format
knowledge_and_skill: 30
grading_guide: - cover content CLO3,CLO4,CLO5
note: in class, by instructor
category: Final exam
type: Final exam
weight: 40.0
duration: 60'
question_type: CLO1,CLO2,CLO3,CLO4,CLO5
no_question: Multiple choices
Marked by Computer
knowledge_and_skill: 50
grading_guide: concepts, algorithms;
all studied slides
note: by exam board,
using computer

Schedule:
session: Course introduction:
Introduce the materials of the courses and how to asses these materials
Introduce the assignments and how to pass the course
learning_outcomes: IT
materials: mat1
session: 1. Introduction to Digital Speech Processing
1.1 The Speech Signal
learning_outcomes: T
session: 1. Introduction to Digital Speech Processing
1.2 The Speech Stack
learning_outcomes: T
session: 1. Introduction to Digital Speech Processing
1.3 Applications of DSP
learning_outcomes: T
session: 1. Introduction to Digital Speech Processing
1.4 Summary
learning_outcomes: T
session: 2. Review of Fundamentals of DSP
2.1 Discrete-Time Signals and Systems
Assignment 1 Introduction
learning_outcomes: T
session: 2. Review of Fundamentals of DSP
2.2 Transform Representation of Signals and Systems
learning_outcomes: T
session: 2. Review of Fundamentals of DSP
2.3 Fundamentals of Digital Filters
learning_outcomes: T
session: 2. Review of Fundamentals of DSP
2.4 Sampling and Summary
learning_outcomes: T
session: 3. Human Speech Production
3.1 The Process of Speech Production
learning_outcomes: T
session: Assignment 1 review
learning_outcomes: T
session: 3. Human Speech Production
3.2 Short-Time Fourier Representation of Speech
learning_outcomes: T
session: 3. Human Speech Production
3.3 Acoustic Phonetics
learning_outcomes: T
session: 3. Human Speech Production
3.4 Distinctive Features of The Phonemes English
learning_outcomes: T
session: 4. Hearing, Auditory Models, and Speech Perception
4.1 The Speech Chain
learning_outcomes: T
session: 4. Hearing, Auditory Models, and Speech Perception
4.2 Anatomy and Function of the Ear
learning_outcomes: T
session: 4. Hearing, Auditory Models, and Speech Perception
4.3 The Perception of Sound
learning_outcomes: T
session: 4. Hearing, Auditory Models, and Speech Perception
4.4 Auditory Models
learning_outcomes: T
session: 4. Hearing, Auditory Models, and Speech Perception
4.5 HSP Experiments
learning_outcomes: T
session: Assignment 1 review
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.1 Short-Time Analysis of Speech
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.2 Short-Time Energy and Short-Time Magnitude
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.3 Short-Time Zero-Crossing Rate
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.4 The Short-Time Autocorrelation Function
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.5 The Modified Short-Time Autocorrelation Function
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.6 The Short-Time Average Magnitude Difference Function
learning_outcomes: T
session: 5. Time-Domain Methods for Speech Processing
5.7 Summary
learning_outcomes: T
session: 6. Frequency-Domain Representations
6.1 Discrete-Time Fourier Analysis
learning_outcomes: T
session: Assignment 1 Evaluation
learning_outcomes: U
session: Progress Test 1
Assignment 2 Introduction
session: 6. Frequency-Domain Representations
6.2 Short-Time Fourier Analysis
learning_outcomes: T
session: 6. Frequency-Domain Representations
6.3 Overlap Addition Method of Synthesis
learning_outcomes: T
session: 6. Frequency-Domain Representations
6.4 Time-Decimated Filter Banks
learning_outcomes: T
session: 6. Frequency-Domain Representations
6.5 Two-Channel Filter Banks
learning_outcomes: T
session: Assignment 2 Review
learning_outcomes: T
session: 7. Linear Predictive Analysis of Speech Signals
7.1 Introduction
learning_outcomes: T
session: 7. Linear Predictive Analysis of Speech Signals
7.2 Basic Principles of Linear Predictive Analysis
learning_outcomes: T
session: 7. Linear Predictive Analysis of Speech Signals
7.3 Computation of the Gain of the Model
learning_outcomes: T
session: 7. Linear Predictive Analysis of Speech Signals
7.4 Frequency Domain Interpretations
learning_outcomes: T
session: 7. Linear Predictive Analysis of Speech Signals
7.5 The Prediction Error Signal
learning_outcomes: T
session: 7. Linear Predictive Analysis of Speech Signals
7.6 Relation of LPA to Lossless Tube Models
learning_outcomes: T
session: 8. Algorithms for Estimating Speech Parameters
8.1 Median Smoothing and Speech Processing
learning_outcomes: T
session: 8. Algorithms for Estimating Speech Parameters
8.2 Speech-Background/Slience Discrimination
learning_outcomes: T
session: 8. Algorithms for Estimating Speech Parameters
8.3 A Bayesian Approach to Voiced Detection
learning_outcomes: T
session: 8. Algorithms for Estimating Speech Parameters
8.4 Formant Estimation
learning_outcomes: T
session: Assignment 2 Review
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.1 Introduction
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.2 Text Analysis
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.3 Evolution of Speech Synthesis Methods
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.3 Early Speech Synthesis Approaches
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.4 Unit Selection Methods
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.5 TTS Future Needs
learning_outcomes: T
session: 9. Text-to-Speech Synthesis Methods
9.6 Visual TTS
learning_outcomes: T
session: Assignment 2 Evaluation
learning_outcomes: T
session: 10. Automatic Speech Recognition and NLU
10.1 Introduction
learning_outcomes: T
session: 10. Automatic Speech Recognition and NLU
10.2 Basic ASR Formulation
learning_outcomes: T
session: 10. Automatic Speech Recognition and NLU
10.3 Overall Speech Recognition Process
learning_outcomes: T
session: 10. Automatic Speech Recognition and NLU
10.4 User Interfaces (UI) and Multimodel UI
learning_outcomes: T
session: Progress Test 2
learning_outcomes: T
session: Summary All Slides, Tests, and Assignments
learning_outcomes: U