Search Engines (SEG301)
Vietnamese title: Công cụ tìm kiếm
Credits: 3
Prerequisites: CSD203, AIL303m

Description:
The theory, creation, and use of text-based search engines are covered in this lecture-based course. The main content includes experimental assessment, representation of information demands and documents, several significant retrieval models, and statistical features of the text. Besides, the course covers typical components of commercial search engines, such as federated search, vertical search, tailored search results, and diversified search results.

Learning outcomes:
Understanding how search engines work: Students should be able to explain the underlying technology behind search engines, including the algorithms used to crawl, index and rank web pages.
Search engine optimization: Students should be able to apply best practices for optimizing web pages to improve their visibility in search engine results pages (SERPs).
Keyword research: Students should be able to conduct keyword research to identify the most relevant and profitable keywords by using popular retrieval models..
Analytics and tracking: Students should be able to use analytics tools to track website traffic, user behavior, and campaign performance.
Solving problems: Students should be able to apply search engine techniques to solve the real-life problems

Student tasks:
- Students must attend at least 80% of contact slots in order to be accepted to the final examination.
- Student is responsible to do all exercises, assignments and labs given by instructor in class or at home and submit on time
- Use laptop in class only for learning purpose
- Promptly access to https://flm.fpt.edu.vn for up-to-date course information

Tools:
No

Materials:
description: Search Engines: Information Retrieval in Practice
author: Bruce Croft, Donald Metzler, Trevor Strohman
publisher: Pearson
isbn: 978-0136072249 (ISDBN-13)
description: Introduction to Information Retrieval
author: Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze
publisher: Cambridge University Press
isbn: 521865719
description: Information Retrieval: Implementing and Evaluating Search Engines
author: Stefan Buttcher, Charles L. A. Clarke, Gordon V. Cormack
publisher: MIT Press
isbn: 978-0262528870 (ISBN-13)
description: Web Information Retrieval
author: Stefano Ceri, Alessandro Bozzon, Marco Brambilla, Emanuele Della Valle, Piero Fraternali, Silvia Quarteroni
publisher: Springer
published_date: 2013
isbn: 978-3642393136 (ISBN-13)
note: https://library.books24x7.com/toc.aspx?bookid=77020
description: Understanding Information Retrieval Systems: Management, Types, and Standards
author: Marcia J. Bates
publisher: Auerbach Publications
published_date: 2011
isbn: 978-1439891964 (ISBN-13)
note: https://library.books24x7.com/toc.aspx?bookid=47168
description: - PowerPoint Slides
description: - Assignments 1
description: - Assignments 2
description: - Assignments 3
description: - Quiz for Progress Tests

Assessments:
category: Assignment 1
type: on-going
weight: 10.0
duration: 45'
question_type: CLO1,CLO3
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content 1-3
note: in class, lecturer select articles and assign to students, by group presentation
category: Assignment 2
type: on-going
weight: 10.0
duration: 45'
question_type: CLO1,CLO3,CLO5
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content 3 - 9
note: in class, lecturer select articles and assign to students, by group presentation
category: Assignment 3
type: on-going
weight: 10.0
duration: 45'
question_type: CLO1,CLO2,CLO4
no_question: Group presentation for the selected article
knowledge_and_skill: 1
grading_guide: - cover content 9 - ends
note: in class, lecturer select articles and assign to students, by group presentation
category: Progress test 1
type: on-going
weight: 15.0
duration: 20'-30'
question_type: CLO1,CLO3,CLO4
no_question: Multiple choicesMarked by Computer or a suitable format
knowledge_and_skill: 20
grading_guide: - cover content chapter 1 -6
note: in class, by instructor
category: Progress test 2
type: on-going
weight: 15.0
duration: 20'-30'
question_type: CLO1,CLO2,CLO4,CLO5
no_question: Multiple choicesMarked by Computer or a suitable format
knowledge_and_skill: 20
grading_guide: - cover content 7 - ends
note: in class, by instructor
category: Final exam
type: Final exam
weight: 40.0
duration: 60'
question_type: CLO 1,2,3,4,5
no_question: Multiple choices
Marked by Computer
knowledge_and_skill: 50
grading_guide: concepts, algorithms;
all studied slides
note: by exam board,
using computer

Schedule:
session: Course introduction:
- Introduce the materials of the courses and how to asses these materials
- Introduce the assignments and how to pass the course
learning_outcomes: IT
materials: mat1
session: 1. Information retieval and search engines:
1.1 Information retrieval
1.2 Search engines
1.3 Search engineers
learning_outcomes: T
materials: drive.google.com
session: 1.4 Web Search
1.5 Other Search Applications 3
1.6 Other IR Applications
learning_outcomes: T
materials: drive.google.com
session: Chapter 1 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 2. Architecture of a seach engine
2.1a Basic IR System Architecture
2.1 Building blocks: Text acquisition, Text transformation, Index Creattion
learning_outcomes: T
materials: drive.google.com
session: 2.1 Building blocks (cont.): User Interaction, Ranking, evaluation
learning_outcomes: T
materials: drive.google.com
session: Chapter 2 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 3 Crawls and Feeds
3.1 What to search.
3.2 Crawling the Web: Retrieving web pages, web crawler, freshness, Crawling, deep web, sitemaps, distributed crawling
learning_outcomes: T
materials: drive.google.com
session: 3.2 Crawling the Web(cont.): Crawling, deep web, sitemaps, distributed crawling
3.3 Crawling documents and email,
learning_outcomes: T
materials: drive.google.com
session: 3.4 docmuneta feeds.
3.5 Conversion problem: character encodings
3.6 Storing the documents: using database system, random access, compression and large files, update, big table
learning_outcomes: T
materials: drive.google.com
session: 3.7 Detecting duplicates
3.8 Remove noise
learning_outcomes: T
materials: drive.google.com
session: Chapter 3 exercises - review
learning_outcomes: T
materials: drive.google.com
session: Assignment 1
learning_outcomes: U
materials: drive.google.com
session: Review Assignment 1
materials: drive.google.com
session: 4 Processing text
4.1 From Words to Terms
4.2 Text statistics
4.3 Document parsing: Tokenizing, stopping, stemming, phrases and N-grams
learning_outcomes: U
materials: drive.google.com
session: 4.4 Document structure and markup.
4.5 Link analysis: anchor text, pagerank, link quality
learning_outcomes: T
materials: drive.google.com
session: 4.6 Information Extraction: Hidden Markov models for extraction
learning_outcomes: T
materials: drive.google.com
session: Chapter 4 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 5. Ranking with indexes.
5.1 Abstract model of raking
5.2 Inverted Indexes: documents, counts, positions, fields and extents, scores, ordering
5.2b Implementing Inverted Indices
learning_outcomes: T
materials: drive.google.com
session: 5.3 Compression: entropy and ambiguity, data encoding, bit-aligned codes, byte-aligned codes,
learning_outcomes: T
materials: drive.google.com
session: 5.3 Compression (cont.): compression in practice, looking ahead, skipping and skip pointers
5.4 Axuiliary structures
learning_outcomes: T
materials: drive.google.com
session: 5.5 Index construction: simple construction, merging, parallelism and distribution, update
learning_outcomes: T
materials: drive.google.com
session: 5.6 Query processing: Document-at-a-time evaluation, term-at-a-time evaluation, optimization techniques,
learning_outcomes: T
materials: drive.google.com
session: 5.6 Query processing (cont.): structured queries, distributed evaluation, caching, Impact Ordering, static index pruning
learning_outcomes: T
materials: drive.google.com
session: Chapter 5 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 6 Queries and interfaces
6.1 Information needs and queries
6.2 Query Transformation and refinement: stopping and stemming revisited, spell checking and sugguestions,
learning_outcomes: T
materials: drive.google.com
session: 6.2 Query Transformation and refinement (cont.): query expansion, relevance feedback, context and personalization
learning_outcomes: T
materials: drive.google.com
session: 6.3 Showing the results: result pages and snippets, advertising and search, clustering the results
6.4 Cross-language search
learning_outcomes: T
materials: drive.google.com
session: Chapter 6 exercises and review
learning_outcomes: U
materials: drive.google.com
session: Progress Test 1 - Review the Test 1
materials: drive.google.com
session: 7. Retrieval Models
7.1 Boolen retrieval, Proximity Ranking and vector space model
7.2 Probabilistic models: Information retrieval as classification, BM25 Ranking Algorithm
learning_outcomes: U
materials: drive.google.com
session: 7.3 Ranking based on Language models: Query likelihood ranking, relevance models and pseudo- relevance feedback
learning_outcomes: T
materials: drive.google.com
session: 7.4 Complex queries and combining evidence: The inference Network model, the Galago Query Language
7.5 Web search
learning_outcomes: T
materials: drive.google.com
session: 7.6 Machine learning and information retrieval: learning to rank. Topic models and vocabulary mismatch
7.7 Application-based models
learning_outcomes: T
materials: drive.google.com
session: Chapter 7 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 8 Search Engine Optimization (SEO)
8.1 The Importance of Search Engine Optimization
8.2 Fundamentals of Search Engine Optimization: Content, Architecture, HTML, Trust, Links, User-Related Factors, Toxins, Vertical Search Engines
learning_outcomes: T
materials: drive.google.com
session: 8.3 Search Engine Optimization and Spam
8.4 The Role of Ranking Updates
8.5 Search Engine Optimization for Special Collections
8.6 The Position of Search Engine Providers
learning_outcomes: T
materials: drive.google.com
session: Chapter 8 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 9. Evaluating search engines
9.1 Why "Evaluate"?
9.2 Evaluation corpus
9.3 Logging
9.4 effectiveness metrices: recall and precision, average and terpolation, focusing on the top documents, using preferences
learning_outcomes: T
materials: drive.google.com
session: 9.5 Efficiency metrics
9.6 Training, testing and statistics: significance tests, setting parameter values, Online testing
learning_outcomes: T
materials: drive.google.com
session: Chapter 9 exercises and review
learning_outcomes: T
materials: drive.google.com
session: Assignment 2
learning_outcomes: T
materials: drive.google.com
session: Review Assignment 2
learning_outcomes: U
materials: drive.google.com
session: 10 Classification and clustering
10.1 classification: Navie bayes, Support vector machines, Evaluation, Classfier and feature selection, spam, sentiment and online advertising
materials: drive.google.com
session: 10 Classification and clustering
10.1 classification: Navie bayes, Support vector machines, Evaluation, Classfier and feature selection, spam, sentiment and online advertising
learning_outcomes: U
materials: drive.google.com
session: 10.2 Clustering: Hierachical and K-meas clustering, K-nearest neighbor clustering, evaluation, how to choose K, clustering and Seach
learning_outcomes: T
materials: drive.google.com
session: 10.2 Clustering: Hierachical and K-meas clustering, K-nearest neighbor clustering, evaluation, how to choose K, clustering and Seach
learning_outcomes: T
materials: drive.google.com
session: Chapter 10 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 11. Social search
11.1 What is Social search?
11.2 User Tags and Manual Indexing: Searching Tags, Inferring Missing Tags,Browsing and Tag Clouds
learning_outcomes: T
materials: drive.google.com
session: 11.3 Searching with Communities: What Is a Community?, Finding Communities, Community-Based Question Answering, Collaborative Searching
learning_outcomes: T
materials: drive.google.com
session: 11.3 Searching with Communities: What Is a Community?, Finding Communities, Community-Based Question Answering, Collaborative Searching
learning_outcomes: T
materials: drive.google.com
session: 11.4 Filtering and Recommending: Document Filtering, Collaborative Filtering
11.5 Peer-to-Peer and Metasearch: Distributed Search, P2P Networks
learning_outcomes: T
materials: drive.google.com
session: Chapter 10 exercises and review
learning_outcomes: T
materials: drive.google.com
session: 12 Beyond bag of words
12.1 Feature-Based Retrieval Models
12.2 Term Dependence Models
learning_outcomes: T
materials: drive.google.com
session: 12.3 Structure Revisited: XML Retrieval, Entity Search
12.4 Longer Questions, Better Answers
12.5 Words, Pictures, and Music
12.6 One Search Fits All?
learning_outcomes: T
materials: drive.google.com
session: Chapter 12 exercises and review
learning_outcomes: U
materials: drive.google.com
session: Progress Test 2 - Review Progress Test 2
learning_outcomes: U
materials: drive.google.com
session: Assignment 3
materials: drive.google.com
session: Review Assignment 3
learning_outcomes: U
materials: drive.google.com
session: Summary All Slides, Tests, and Assignments
learning_outcomes: U
materials: drive.google.com