چگونه یک "سیستم بازیابی اطلاعات " بسازم؟  - کاربرد "موتورهای جستجو،جستجوگرهای تحت وب،خزنده های وب و..." -  کتاب "مقدمه بر بازیابی اطلاعات" (سرفصل ها)

سلامبه نام خدامقدمه ای بر بازیابی اطلاعاتAn Introduction to Information Retrieval Christopher D. Manning Prabhakar Raghavan Hinrich Schütze Cambridge University Press Cambridge, England DRAFT! DO NOT DISTRIBUTE WITHOUT PRIOR PERMISSION © 2009 Cambridge University Press By Christopher D. Manning, Prabhakar Raghavan & Hinrich Schütze Printed on April 1, 2009 Website: http://www.informationretrieval.org/ Comments, corrections, and other feedback most welcome at: informationretrieval@yahoogroups.comBrief Contents1 Boolean retrieval 12 The term vocabulary and postings lists 193 Dictionaries and tolerant retrieval 494 Index construction 675 Index compression 856 Scoring, term weighting and the vector space model 1097 Computing scores in a complete search system 1358 Evaluation in information retrieval 1519 Relevance feedback and query expansion 17710 XML retrieval 19511 Probabilistic information retrieval 21912 Language models for information retrieval 23713 Text classification and Naive Bayes 25314 Vector space classification 28915 Support vector machines and machine learning on documents 31916 Flat clustering 34917 Hierarchical clustering 37718 Matrix decompositions and latent semantic indexing 40319 Web search basics 42120 Web crawling and indexes 44321 Link analysis 461ContentsContentsList of TablesList of FiguresTable of Notationsi) Preface i.1 Book organization and course developmenti.2 Prerequisites i.3 Book layouti.4 Acknowledgmentsi.5 Web and contact informationA) Boolean retrievalA.1An example information retrieval problemA.2 A first take at building an inverted indexA.3 Processing Boolean queriesA.4The extended Boolean model versus ranked retrievalA.5 References and further readingB)The term vocabulary and postings listsB.1Document delineation and character sequence decodingB.1.1 Obtaining the character sequence in a documentB.1.2 Choosing a document unitB.2 Determining the vocabulary of termsB.2.1 TokenizationB.2 .2 Dropping common terms: stop wordsB.2.3 Normalization (equivalence classing of terms)B.2.4 Stemming and lemmatizationB.3 Faster postings list intersection via skip pointersB.4 Positional postings and phrase queriesB.4.1 Biword indexesB.4.2 Positional indexesB.4.3 Combination schemes References and further readingC )Dictionaries and tolerant retrievalC.1 Search structures for dictionariesC.2 Wildcard queriesC.2.1 General wildcard queriesC.2.1.1 Permuterm indexesC.2.2 k-gram indexes for wildcard queriesC.3 Spelling correction C.3.1 Implementing spelling correctionC.3.2 Forms of spelling correctionC.3.3 Edit distanceC.3.4 k-gram indexes for spelling correctionC.3.5 Context sensitive spelling correctionC.4 Phonetic correctionC.5 References and further readingD) Index constructionD.1 Hardware basicsD.2 Blocked sort-based indexingD.3 Single-pass in-memory indexingD.4 Distributed indexingD.5 Dynamic indexingD.6 Other types of indexesD.7 References and further readingE) Index compressionE.1 Statistical properties of terms in information retrievalE.1.1 Heaps law: Estimating the number of termsE.1.2 Zip law: Modeling the distribution of termsE.2 Dictionary compressionE.2.1 Dictionary as a stringE.2.2Blocked storageE.3 Postings file compression E.3.1 Variable byte codesE.3.2Gamma codesE.4 References and further readingF) Scoring, term weighting and the vector space modelF.1 Parametric and zone indexesF.1.1Weighted zone scoringF.1.2 Learning weightsF.1.3 The optimal weight gF.2 Term frequency and weightingF.2 .1 Inverse document frequencyF.2.2Tf-idf weightingF.3 The vector space model for scoringF.3.1 Dot productsF.3.2 Queries as vectorsF.3.3 Computing vector scoresF.4 Variant tf-idf functionsF.4.1 Sublinear tf scalingF.4.2 Maximum tf normalizationF.4.3 Document and query weighting schemesF.4.4 Pivoted normalized document lengthF.5 References and further readingG) Computing scores in a complete search systemG.1 Efficient scoring and rankingG.1.1 Inexact top K document retrievalG.1.2 Index eliminationG.1.3 Champion listsG.1.4 Static quality scores and orderingG.1.5 Impact orderingG.1.6 Cluster pruningG.2 Components of an information retrieval systemG.2.1 Tiered indexesG.2.2 Query-term proximityG.2.3 Designing parsing and scoring functionsG.2.4 Putting it all togetherG.3 Vector space scoring and query operator interactionG.3.1 Boolean retrievalG.3.2 Wildcard queriesG.3.3 Phrase queriesG.3.4 References and further readingH) Evaluation in information retrievalH.1 Information retrieval system evaluationH.2 Standard test collectionsH.3 Evaluation of unranked retrieval setsH.4 Evaluation of ranked retrieval resultsH.5 Assessing relevanceH.5.1 Critiques and justifications of the concept of relevanceH.6 A broader perspective: System quality and user utilityH.6 .1 System issuesH.6.2User utilityH.6.3Refining a deployed systemH.7Results snippetsH.8References and further readingJ) Relevance feedback and query expansionJ.1 Relevance feedback and pseudo relevance feedbackJ.1.1 The Rocchio algorithm for relevance feedbackJ.1.2 Probabilistic relevance feedbackJ.1.3 When does relevance feedback work?J.1.4 Relevance feedback on the webJ.1.5 Evaluation of relevance feedback strategiesJ.1.6Pseudo relevance feedbackJ.1.7 Indirect relevance feedbackJ.1.8 SummaryJ.2 Global methods for query reformulationJ.2.1 Vocabulary tools for query reformulationJ.2.2 Query expansionJ.2.3 Automatic thesaurus generationJ.3 References and further readingK) XML retrievalK.1 Basic XML conceptsK.2 Challenges in XML retrievalK.3 A vector space model for XML retrievalK.4 Evaluation of XML retrievalK.5 Text-centric vs. data-centric XML retrievalK.6 References and further readingK.7 ExercisesM) Probabilistic information retrievalM.1 Review of basic probability theoryM.2 The Probability Ranking PrincipleM.2 .1 The 1/0 loss caseM.2.2The PRP with retrieval costsM.3 The Binary Independence Model M.3.1 Deriving a ranking function for query termsM.3.2 Probability estimates in theoryM.3.3 Probability estimates in practiceM.3.4Probabilistic approaches to relevance feedbackM.4 An appraisal and some extensionsM.4 .1 An appraisal of probabilistic modelsM.4.2 Tree-structured dependencies between termsM.4.3 Okapi BM25: a non-binary modelM.4.4 Bayesian network approaches to IRM.5 References and further readingN) Language models for information retrievalN.1 Language models N.1.1 Finite automata and language modelsN.1.2 Types of language modelsN.1.3 Multinomial distributions over wordsN.2 The query likelihood modelN.2.1 Using query likelihood language models in IRN.2.2 Estimating the query generation probabilityN.2.3 Ponte and Croft ExperimentsN.3 Language modeling versus other approaches in IRN.4 Extended language modeling approachesN.5References and further readingO) Text classification and Naive BayesO.1 The text classification problemO.2 Naive Bayes text classificationO.2.1 Relation to multinomial unigram language modelO.3 The Bernoulli modelO.4 Properties of Naive BayesO.4.1 A variant of the multinomial modelO.5 Feature selectionO.5.1 Mutual informationO.5.2 X^2 Feature selectionChi2 Feature selectionO.5.2.1 Assessing X^2 as a feature selection method Assessing chi-square as a feature selection methodO.5.3 Frequency-based feature selectionO.5.4 Feature selection for multiple classifiersO.5.5 Comparison of feature selection methodsO.6 Evaluation of text classificationO.7 References and further readingP) Vector space classificationP.1 Document representations and measures of relatedness in vector spacesP.2 Rocchio classificationP.3 k nearest neighborP.3.1 Time complexity and optimality of kNNP.4 Linear versus nonlinear classifiersP.5 Classification with more than two classesP.6 The bias-variance tradeoffP.7 References and further readingP.8 ExercisesR) Support vector machines and machine learning on documentsR.1 Support vector machines: The linearly separable caseR.2 Extensions to the SVM modelR.2.1 Soft margin classificationR.2.2 Multiclass SVMsR.2.3 Nonlinear SVMsR.2.4 Experimental resultsR.3 Issues in the classification of text documentsR.3.1 Choosing what kind of classifier to useR.3.2 Improving classifier performanceR.3.2.1 Large and difficult category taxonomiesR.3.2.2 Features for textR.3.2.3Document zones in text classificationR.4 Machine learning methods in ad hoc information retrieval R.4.1 A simple example of machine-learned scoringR.4.2Result ranking by machine learningR.5 References and further readingS) Flat clusteringS.1 Clustering in information retrievalS.2 Problem statement S.2.1 Cardinality - the number of clustersS.3 Evaluation of clusteringS.4 K-meansS.4.1 Cluster cardinality in K-meansS.5 Model-based clusteringS.6 References and further readingS.7 ExercisesQ) Hierarchical clustering Q.1 Hierarchical agglomerative clusteringQ.2 Single-link and complete-link clusteringQ.2.1 Time complexity of HACQ.3 Group-average agglomerative clusteringQ.4 Centroid clusteringQ.5 Optimality of HACQ.6 Divisive clusteringQ.7 Cluster labelingQ.8 Implementation notesQ.9 References and further readingQ.10 ExercisesT) Matrix decompositions and latent semantic indexingT.1 Linear algebra review T.1.1 Matrix decompositionsT.2 Term-document matrices and singular value decompositionsT.3Low-rank approximationsT.4 Latent semantic indexingT.5 References and further readingW) Web search basicsW.1 Background and historyW.2 Web characteristics W.2.1 The web graphW.2.2 SpamW.3 Advertising as the economic modelW.4The search user experienceW.4.1 User query needsW.5 Index size and estimationW.6 Near-duplicates and shinglingW.7 References and further readingY) Web crawling and indexesY.1 OverviewY.1.1 Features a crawler must provideY.1.2Features a crawler should provideY.2 CrawlingY.2.1Crawler architectureY.2.1.1 Distributing the crawlerY.2.2 DNS resolutionY.2.3The URL frontierY.3 Distributing indexesY.3Connectivity serversY.4 References and further readingV) Link analysisV.1 The Web as a graphV.1.1 Anchor text and the web graphV.2 PageRankV.2.1 Markov chainsV.2.2 The PageRank computationV.2.3 Topic-specific PageRankV.3 Hubs and AuthoritiesV.3.1Choosing the subset of the WebV.4References and further readingBibliographyIndexAbout this document ...منبع https://nlp.stanford.edu/IR-book/html/htmledition/irbook.html موفق باشید به امید خدا