[major=AI | course_id=TMG301 | syllabus_id=11561 | semester=5 | combo=AI17_COM2.1 | title=Text Mining]
Course ID: TMG301
Title EN: Text Mining
Title VI: Khai thác văn bản
Credits: 3.0
Prerequisites: CSD203, AIL303m
Description: This course will cover the primary techniques for mining and analyzing text data to discover interesting patterns, extract practical knowledge, and support decision-making, emphasizing statistical approaches that can be generally applied to arbitrary text data in any natural language with no or minimum human effort. Detailed analysis of text data requires understanding natural language text, which is a difficult task for computers
Student tasks: - Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible to do all exercises, assignments and labs given by instructor in class or at home and submit on time
- Use laptop in class only for learning purpose
- Promptly access to the FU FLM at https://flm.fpt.edu.vn/ for up-to-date course information
Tools: No
Course type: traditional
Decision: 765/QĐ-ĐHFPT dated 07/29/2024
Approved date: 2024-07-29
CLO1: Students will develop a solid understanding of the fundamental concepts and principles of text mining, including techniques for extracting, transforming, and analyzing textual data.
[1] category=Assignment 1; type=on-going; weight=15.0; duration=45'; question_type=CLO1, CLO2; grading_guide=- cover content 1 - 8; completion_criteria=; clos=>0
[2] category=Assignment 2; type=on-going; weight=15.0; duration=45'; question_type=CLO1, CLO2, CLO3, CLO4, CLO5; grading_guide=- cover content 9 - 18; completion_criteria=; clos=>0
[3] category=Progress test 1; type=on-going; weight=15.0; duration=20'-30'; question_type=CLO1, CLO2, CLO3, CLO4, CLO5; grading_guide=- cover content 1 - 9; completion_criteria=; clos=>0
[4] category=Progress test 2; type=on-going; weight=15.0; duration=20'-30'; question_type=CLO1, CLO2, CLO3, CLO4, CLO5; grading_guide=- cover content 9 - 18; completion_criteria=; clos=>0
[5] category=Final exam; type=Final exam; weight=40.0; duration=60'; question_type=CLO1, CLO2, CLO3, CLO4, CLO5; grading_guide=concepts, algorithms;
all studied slides; completion_criteria=1) Every on-going assessment component > 0
2) Final Exam Score >= 4 & Final Result >= 5

Bonus score for accepted paper (email accepted/ published):
ISI/Scopus Q1, Q2: 5
ISI/Scopus Q3, Q4: 3
Scopus Q1, Q2: 3
Scopus Q3, Q4: 2

The accepted paper must be related to topics in AI17_COM2 and be assessed by the in-class Lecturer.

Source to check Scopus ranking: https://www.scimagojr.com/
Source to check ISI: https://mjl.clarivate.com; clos=4
[1] desc=Applied Text Analysis with Python: Enabling Language-Aware Data Products with Machine Learning; author=Benjamin Bengfort, Rebecca Bilbro, Tony Ojeda; publisher=O'Reilly Media; edition=; isbn=978-1491963043 (ISBN-13); main=; online=; note=
[2] desc=Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit; author=Steven Bird, Ewan Klein, Edward Loper; publisher=O'Reilly Media; edition=; isbn=978-0596516499 (ISBN-13); main=; online=; note=
[3] desc=Speech and Language Processing; author=Daniel Jurafsky, James Martin; publisher=Prentice Hall; edition=; isbn=978-0131873216 (ISBN-13); main=; online=; note=
[4] desc=Natural Language Processing in Action: Understanding, Analyzing, and Generating Text with Python; author=Hobson Lane, Cole Howard and Hannes Max Hapke; publisher=Manning; edition=; isbn=978-1617294631 (ISBN-13); main=True; online=True; note=https://library.books24x7.com/toc.aspx?bookid=147295
[5] desc=Text Analytics with Python: A Practitioner's Guide to Natural Language Processing; author=Dipanjan Sarkar; publisher=Apress; edition=Second Edition; isbn=978-1484243534 (ISBN-13); main=True; online=True; note=https://library.books24x7.com/toc.aspx?bookid=146229
[6] desc=- PowerPoint Slides; author=; publisher=; edition=; isbn=; main=; online=; note=
[7] desc=- Assignments 1; author=; publisher=; edition=; isbn=; main=; online=; note=
[8] desc=- Assignments 2; author=; publisher=; edition=; isbn=; main=; online=; note=
[9] desc=- Quiz for Progress Tests; author=; publisher=; edition=; isbn=; main=; online=; note=
Session 1: Course introduction:
- Introduce the materials of the courses and how to asses these materials
- Introduce the assignments and how to pass the course
Topic: Offline; Teaching type: CLO1; LO: IT
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: mat1
Download: Read Slide and and Seft-Review

Session 2: 1. Regular expressions, text normalization, sdit distance
1.1 regular expressions, words, corpora
Topic: Offline; Teaching type: CLO1; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 1. Regular expressions, text normalization, sdit distance
1.2 Text normalization, minimum edit distance
Topic: Offline; Teaching type: CLO1; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 2. N-gram language models
2.1 N-Grams, evaluating language models,
Topic: Offline; Teaching type: CLO1; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
CLO2: Students can use machine learning/deep learning (neural networks, neural language models, recurrent neural networks, LSMT) for specific text-mining applications.
Session 1: 2. N-gram language models
2.2 sampling sentences from a language model, generalization and zeros, Smoothing
Topic: Offline; Teaching type: CLO1; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 2. N-gram language models
2.3 Knerser-Ney smoothing, perplexity's relation to entropy
- Assignment 1 introduction
Topic: Offline; Teaching type: CLO1; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 3. Naïve Bayes, Text classification, and Sentiment
3.1 Naïve Bayes classifiers, Training the Naive Bayes Classifier
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 3. Naïve Bayes, Text classification, and Sentiment
3.2 Optimizing for sentiment analysis, Naive Bayes for other text classification tasks,
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
CLO3: Students will explore the methods and algorithms used for text classification and clustering tasks. They will learn to apply techniques like Naive Bayes, Support Vector Machines, and k-means clustering.
Session 1: 3. Naïve Bayes, Text classification, and Sentiment
3.3 Naïve Bayes as a language model, Evaluation: Precision, Recall, F-measure,
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 3. Naïve Bayes, Text classification, and Sentiment
3.4 test sets and cross-validation, statistical significance testing
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: Assignment 1 support
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 4. Logistic regression
4.1 Review of logistic regression (Sigmoid function, classification with logistic regression)
4.2 Multinomial logistic regression
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
CLO4: Students will learn about different methods for representing text data, such as bag-of-words, term frequency-inverse document frequency (TF-IDF), and word embeddings. They will understand the strengths and limitations of each approach.
Session 1: 4. Logistic regression
4.3 Learning in Logistic Regression , The cross-entropy loss function, Gradient Descent, Regularization
Topic: Offline; Teaching type: CLO1, CLO2; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slides and and Seft-Review

Session 2: 4. Logistic regression
4.4 Interpreting models, deriving the gradient equation
Topic: Offline; Teaching type: CLO1, CLO2; LO: 
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Assignment 1

Session 3: 5. Vector semantics and embeddings
5.1 Lexical semantics, vector semantics,
Topic: Offline; Teaching type: CLO1, CLO4; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Review Assignment 1

Session 4: 5. Vector semantics and embeddings
5.2 words and vectors, cosine for measuring similarity
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
CLO5: Students will be aware of the popular applications in text mining, such as machine translation, question answering, and chatbots to design text mining-related applications in real-world
Session 1: 5. Vector semantics and embeddings
5.3 TF-IDF: weighting terms in the vector, pointwise mutual information
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 5. Vector semantics and embeddings
5.4 Word2vec
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 5. Vector semantics and embeddings
5.5 Visualizing Embeddings, Semantic properties of embeddings, Bias and Embeddings, Evaluating Vector Models
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: Assignment 1 support
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 6. Clustering for Text Similarity
6.1 Unsupervised Learning on Text
6.2 Clustering by Document Similarity
Topic: Offline; Teaching type: CLO1, CLO3; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 6. Clustering for Text Similarity
6.3 Modeling Document Topics
Topic: Offline; Teaching type: CLO1, CLO3; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 7. RNNs and LSTMs
7.1 Recurrent neural networks, RNNs as Language models, RNNs for other NL tasks
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 7. RNNs and LSTMs
7.2 The LSTM, common RNN NL architectures, attention
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 8. Transformers and pretrained language models
8.1 self-attention networks: transformers, transformers as language models, sampling, beam search
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 8. Transformers and pretrained language models
8.2 pretraining large language models, language models for zero- shot learning, potential harms from language models
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 9. Fine-tuning and masked language models
9.1 Bidirectional transformer encoders, training bidirectional encoders
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 9. Fine-tuning and masked language models
9.2 Transfer learning through fine-tuning, training corpora
Topic: Offline; Teaching type: CLO1, CLO2; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: Assignment 1 evaluation
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slides and and Seft-Review

Session 2: Progress test 1
Assignment 2 introduction
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: 
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Test 1

Session 3: 10. Machine translation
10.1 Leanguage divergeneces and typology, machine translation using encoder-decoder, details of the encoder-decoder model
Topic: Offline; Teaching type: CLO1, CLO5; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Review Test 1

Session 4: 10. Machine translation
10.2 Translating in low-resource situations, machine traslation evaluation, bias and ethical issues
Topic: Offline; Teaching type: CLO1, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 11. Chatbot and Dialogue systems
11.1 Properties of human conversation, chatbots
Topic: Offline; Teaching type: CLO1, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 11. Chatbot and Dialogue systems
11.2 GUS: simple frame-based diaglogue systems, The dialogue-state architecture,
Topic: Offline; Teaching type: CLO1, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 11. Chatbot and Dialogue systems
11.3 Evaluating dialogue systems, dialogue system design
Topic: Offline; Teaching type: CLO1, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: Assignment 2 support
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 12. Context-free grammars and constituency parsing
12.1 Constituency, context-free grammars, treebanks, grammar equivalence and normal form
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 12. Context-free grammars and constituency parsing
12.2 Ambiguity, a dynamic programming approach
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 12. Context-free grammars and constituency parsing
12.3 Span-based neural constituency parsing, evaluating parsers
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 13. Dependency parsing
13.1 Dependency relations, transition-based dependency parsing
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 13. Dependency parsing
13.2 Graph-based dependency parsing, evaluation
Exercise
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: Assignment 2 support
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Assignment 2

Session 3: 14. Logical representations of sentence meaning
14.1 Model-Theoretic semantics, first-order logic
14.2 event and state representations, description logics
Topic: Offline; Teaching type: CLO1, CLO4; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Review Assignment 2

Session 4: 15. Relation and Event extraction
15.1 Relation extraction Algorithms,
Topic: Offline; Teaching type: CLO1, CLO4; LO: 
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 15. Relation and Event extraction
15.2 Extracting events, template filling
Topic: Offline; Teaching type: CLO1, CLO4; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 16. Time and temporal reasoning
16.1 Representing time, representing aspect
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 16. Time and temporal reasoning
16.2 Temporally annotated datasets: time bank, automatic temporal analysis
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: Assignment 2 support
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 17. Word senses and WordNet
17.1 Word senses, relations between senses, WordNet: a database of lexical relations
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: 17. Word senses and WordNet
17.2 Word sense Disambguation, Alternate WSD algorithms and tasks
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 17. Word senses and WordNet
17.3 Using thesauruses to improve embeddings, word sense induction
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 18. Lexicons for sentiment, affect, and connotation
18.1 Defining emotion, available sentiment and affect lexicons, creating affect lexicons by human labeling
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review
Session 1: 18. Lexicons for sentiment, affect, and connotation
18.2 Semi-supervised induction of affect lexicons
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 2: Assignment 2 support
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 3: 18. Lexicons for sentiment, affect, and connotation
18.3 supervised learning of word sentiment
Topic: Offline; Teaching type: CLO1, CLO4; LO: T
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slide and and Seft-Review

Session 4: 18. Lexicons for sentiment, affect, and connotation
18.4 using lexicons for sentiment recognition, using lexicons for affect recognition
Topic: Offline; Teaching type: CLO1, CLO4; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Read Slides and and Seft-Review
Session 1: 18. Lexicons for sentiment, affect, and connotation
18.5 Lexicon-based methods for Entity-Centric Affect, Connotation Frames
Topic: Offline; Teaching type: CLO1, CLO4; LO: U
ITU: Course Implementation
- Syllabus TMG301m
- Slides
Materials: drive.google.com
Download: Review Test 2

Session 2: Progress Test 2 - Review Progress Test 2
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: 
ITU: Content 16-28
Progress Test 2
Materials: drive.google.com
Download: Review Test 2

Session 3: Assignment 2 evaluation
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: U
ITU: Contents 23-28
Materials: drive.google.com
Download: Assignment 3

Session 4: Summary All Slides, Tests, and Assignments
Topic: Offline; Teaching type: CLO1, CLO2, CLO3, CLO4, CLO5; LO: U
ITU: All Contents
Materials: drive.google.com
Download: Review and Group discussion