Text Mining (TMG301)
Vietnamese title: Khai thác văn bản
Credits: 3
Prerequisites: CSD203, AIL303m

Description:
This course will cover the primary techniques for mining and analyzing text data to discover interesting patterns, extract practical knowledge, and support decision-making, emphasizing statistical approaches that can be generally applied to arbitrary text data in any natural language with no or minimum human effort. Detailed analysis of text data requires understanding natural language text, which is a difficult task for computers

Learning outcomes:
Students will develop a solid understanding of the fundamental concepts and principles of text mining, including techniques for extracting, transforming, and analyzing textual data.
Students can use machine learning/deep learning (neural networks, neural language models, recurrent neural networks, LSMT) for specific text-mining applications.
Students will explore the methods and algorithms used for text classification and clustering tasks. They will learn to apply techniques like Naive Bayes, Support Vector Machines, and k-means clustering.
Students will learn about different methods for representing text data, such as bag-of-words, term frequency-inverse document frequency (TF-IDF), and word embeddings. They will understand the strengths and limitations of each approach.
Students will be aware of the popular applications in text mining, such as machine translation, question answering, and chatbots to design text mining-related applications in real-world

Student tasks:
- Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible to do all exercises, assignments and labs given by instructor in class or at home and submit on time
- Use laptop in class only for learning purpose
- Promptly access to the FU FLM at https://flm.fpt.edu.vn/ for up-to-date course information

Tools:
No

Materials:
description: Applied Text Analysis with Python: Enabling Language-Aware Data Products with Machine Learning
author: Benjamin Bengfort, Rebecca Bilbro, Tony Ojeda
publisher: O'Reilly Media
published_date: 2018
isbn: 978-1491963043 (ISBN-13)
description: Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit
author: Steven Bird, Ewan Klein, Edward Loper
publisher: O'Reilly Media
published_date: 2009
isbn: 978-0596516499 (ISBN-13)
description: Speech and Language Processing
author: Daniel Jurafsky, James Martin
publisher: Prentice Hall
published_date: 2008
isbn: 978-0131873216 (ISBN-13)
description: Natural Language Processing in Action: Understanding, Analyzing, and Generating Text with Python
author: Hobson Lane, Cole Howard and Hannes Max Hapke
publisher: Manning
published_date: 2019
isbn: 978-1617294631 (ISBN-13)
note: https://library.books24x7.com/toc.aspx?bookid=147295
description: Text Analytics with Python: A Practitioner's Guide to Natural Language Processing
author: Dipanjan Sarkar
publisher: Apress
published_date: 2019
edition: Second Edition
isbn: 978-1484243534 (ISBN-13)
note: https://library.books24x7.com/toc.aspx?bookid=146229
description: - PowerPoint Slides
description: - Assignments 1
description: - Assignments 2
description: - Quiz for Progress Tests

Assessments:
category: Assignment 1
type: on-going
weight: 15.0
duration: 45'
question_type: CLO1, CLO2
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content 1 - 8
note: in class, the lecturer selects articles and assign them to students, by group presentation
category: Assignment 2
type: on-going
weight: 15.0
duration: 45'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content 9 - 18
note: in class, the lecturer selects articles and assign them to students, by group presentation
category: Progress test 1
type: on-going
weight: 15.0
duration: 20'-30'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5
no_question: Multiple choicesMarked by Computer or a suitable format
knowledge_and_skill: 20
grading_guide: - cover content 1 - 9
note: in class, by instructor
category: Progress test 2
type: on-going
weight: 15.0
duration: 20'-30'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5
no_question: Multiple choicesMarked by Computer or a suitable format
knowledge_and_skill: 20
grading_guide: - cover content 9 - 18
note: in class, by instructor
category: Final exam
type: Final exam
weight: 40.0
duration: 60'
question_type: CLO1, CLO2, CLO3, CLO4, CLO5
no_question: Multiple choices
Marked by Computer
knowledge_and_skill: 50
grading_guide: concepts, algorithms;
all studied slides
note: by exam board,
using computer

Schedule:
session: Course introduction:
- Introduce the materials of the courses and how to asses these materials
- Introduce the assignments and how to pass the course
learning_outcomes: IT
materials: mat1
session: 1. Regular expressions, text normalization, sdit distance
1.1 regular expressions, words, corpora
learning_outcomes: T
materials: drive.google.com
session: 1. Regular expressions, text normalization, sdit distance
1.2 Text normalization, minimum edit distance
learning_outcomes: T
materials: drive.google.com
session: 2. N-gram language models
2.1 N-Grams, evaluating language models,
learning_outcomes: T
materials: drive.google.com
session: 2. N-gram language models
2.2 sampling sentences from a language model, generalization and zeros, Smoothing
learning_outcomes: T
materials: drive.google.com
session: 2. N-gram language models
2.3 Knerser-Ney smoothing, perplexity's relation to entropy
- Assignment 1 introduction
learning_outcomes: T
materials: drive.google.com
session: 3. Naïve Bayes, Text classification, and Sentiment
3.1 Naïve Bayes classifiers, Training the Naive Bayes Classifier
learning_outcomes: T
materials: drive.google.com
session: 3. Naïve Bayes, Text classification, and Sentiment
3.2 Optimizing for sentiment analysis, Naive Bayes for other text classification tasks,
learning_outcomes: T
materials: drive.google.com
session: 3. Naïve Bayes, Text classification, and Sentiment
3.3 Naïve Bayes as a language model, Evaluation: Precision, Recall, F-measure,
learning_outcomes: T
materials: drive.google.com
session: 3. Naïve Bayes, Text classification, and Sentiment
3.4 test sets and cross-validation, statistical significance testing
learning_outcomes: T
materials: drive.google.com
session: Assignment 1 support
learning_outcomes: T
materials: drive.google.com
session: 4. Logistic regression
4.1 Review of logistic regression (Sigmoid function, classification with logistic regression)
4.2 Multinomial logistic regression
learning_outcomes: T
materials: drive.google.com
session: 4. Logistic regression
4.3 Learning in Logistic Regression , The cross-entropy loss function, Gradient Descent, Regularization
learning_outcomes: U
materials: drive.google.com
session: 4. Logistic regression
4.4 Interpreting models, deriving the gradient equation
materials: drive.google.com
session: 5. Vector semantics and embeddings
5.1 Lexical semantics, vector semantics,
learning_outcomes: U
materials: drive.google.com
session: 5. Vector semantics and embeddings
5.2 words and vectors, cosine for measuring similarity
learning_outcomes: T
materials: drive.google.com
session: 5. Vector semantics and embeddings
5.3 TF-IDF: weighting terms in the vector, pointwise mutual information
learning_outcomes: T
materials: drive.google.com
session: 5. Vector semantics and embeddings
5.4 Word2vec
learning_outcomes: T
materials: drive.google.com
session: 5. Vector semantics and embeddings
5.5 Visualizing Embeddings, Semantic properties of embeddings, Bias and Embeddings, Evaluating Vector Models
learning_outcomes: T
materials: drive.google.com
session: Assignment 1 support
learning_outcomes: T
materials: drive.google.com
session: 6. Clustering for Text Similarity
6.1 Unsupervised Learning on Text
6.2 Clustering by Document Similarity
learning_outcomes: T
materials: drive.google.com
session: 6. Clustering for Text Similarity
6.3 Modeling Document Topics
learning_outcomes: T
materials: drive.google.com
session: 7. RNNs and LSTMs
7.1 Recurrent neural networks, RNNs as Language models, RNNs for other NL tasks
learning_outcomes: T
materials: drive.google.com
session: 7. RNNs and LSTMs
7.2 The LSTM, common RNN NL architectures, attention
learning_outcomes: T
materials: drive.google.com
session: 8. Transformers and pretrained language models
8.1 self-attention networks: transformers, transformers as language models, sampling, beam search
learning_outcomes: T
materials: drive.google.com
session: 8. Transformers and pretrained language models
8.2 pretraining large language models, language models for zero- shot learning, potential harms from language models
learning_outcomes: T
materials: drive.google.com
session: 9. Fine-tuning and masked language models
9.1 Bidirectional transformer encoders, training bidirectional encoders
learning_outcomes: T
materials: drive.google.com
session: 9. Fine-tuning and masked language models
9.2 Transfer learning through fine-tuning, training corpora
learning_outcomes: T
materials: drive.google.com
session: Assignment 1 evaluation
learning_outcomes: U
materials: drive.google.com
session: Progress test 1
Assignment 2 introduction
materials: drive.google.com
session: 10. Machine translation
10.1 Leanguage divergeneces and typology, machine translation using encoder-decoder, details of the encoder-decoder model
learning_outcomes: U
materials: drive.google.com
session: 10. Machine translation
10.2 Translating in low-resource situations, machine traslation evaluation, bias and ethical issues
learning_outcomes: T
materials: drive.google.com
session: 11. Chatbot and Dialogue systems
11.1 Properties of human conversation, chatbots
learning_outcomes: T
materials: drive.google.com
session: 11. Chatbot and Dialogue systems
11.2 GUS: simple frame-based diaglogue systems, The dialogue-state architecture,
learning_outcomes: T
materials: drive.google.com
session: 11. Chatbot and Dialogue systems
11.3 Evaluating dialogue systems, dialogue system design
learning_outcomes: T
materials: drive.google.com
session: Assignment 2 support
learning_outcomes: T
materials: drive.google.com
session: 12. Context-free grammars and constituency parsing
12.1 Constituency, context-free grammars, treebanks, grammar equivalence and normal form
learning_outcomes: T
materials: drive.google.com
session: 12. Context-free grammars and constituency parsing
12.2 Ambiguity, a dynamic programming approach
learning_outcomes: T
materials: drive.google.com
session: 12. Context-free grammars and constituency parsing
12.3 Span-based neural constituency parsing, evaluating parsers
learning_outcomes: T
materials: drive.google.com
session: 13. Dependency parsing
13.1 Dependency relations, transition-based dependency parsing
learning_outcomes: T
materials: drive.google.com
session: 13. Dependency parsing
13.2 Graph-based dependency parsing, evaluation
Exercise
learning_outcomes: T
materials: drive.google.com
session: Assignment 2 support
learning_outcomes: T
materials: drive.google.com
session: 14. Logical representations of sentence meaning
14.1 Model-Theoretic semantics, first-order logic
14.2 event and state representations, description logics
learning_outcomes: U
materials: drive.google.com
session: 15. Relation and Event extraction
15.1 Relation extraction Algorithms,
materials: drive.google.com
session: 15. Relation and Event extraction
15.2 Extracting events, template filling
learning_outcomes: U
materials: drive.google.com
session: 16. Time and temporal reasoning
16.1 Representing time, representing aspect
learning_outcomes: T
materials: drive.google.com
session: 16. Time and temporal reasoning
16.2 Temporally annotated datasets: time bank, automatic temporal analysis
learning_outcomes: T
materials: drive.google.com
session: Assignment 2 support
learning_outcomes: T
materials: drive.google.com
session: 17. Word senses and WordNet
17.1 Word senses, relations between senses, WordNet: a database of lexical relations
learning_outcomes: T
materials: drive.google.com
session: 17. Word senses and WordNet
17.2 Word sense Disambguation, Alternate WSD algorithms and tasks
learning_outcomes: T
materials: drive.google.com
session: 17. Word senses and WordNet
17.3 Using thesauruses to improve embeddings, word sense induction
learning_outcomes: T
materials: drive.google.com
session: 18. Lexicons for sentiment, affect, and connotation
18.1 Defining emotion, available sentiment and affect lexicons, creating affect lexicons by human labeling
learning_outcomes: T
materials: drive.google.com
session: 18. Lexicons for sentiment, affect, and connotation
18.2 Semi-supervised induction of affect lexicons
learning_outcomes: T
materials: drive.google.com
session: Assignment 2 support
learning_outcomes: T
materials: drive.google.com
session: 18. Lexicons for sentiment, affect, and connotation
18.3 supervised learning of word sentiment
learning_outcomes: T
materials: drive.google.com
session: 18. Lexicons for sentiment, affect, and connotation
18.4 using lexicons for sentiment recognition, using lexicons for affect recognition
learning_outcomes: U
materials: drive.google.com
session: 18. Lexicons for sentiment, affect, and connotation
18.5 Lexicon-based methods for Entity-Centric Affect, Connotation Frames
learning_outcomes: U
materials: drive.google.com
session: Progress Test 2 - Review Progress Test 2
materials: drive.google.com
session: Assignment 2 evaluation
learning_outcomes: U
materials: drive.google.com
session: Summary All Slides, Tests, and Assignments
learning_outcomes: U
materials: drive.google.com