# Information Retrieval

## Summary


Information Retrieval (IR) refers to finding out relevant information from any kind of data. It may be audio, video, document, article or an image. Traditional ways of information retrieval consist of breaking down data into subsets or clusters across dimensions and finding relevant information according to problem statement.

There are various ways to retrieve information from any unstructured data. It started with rule-based models. More recently, neural networks are being used.

The way humans understand is far more complex than how machines understand. Machines follow the logic of finding out similarities, differences or links with the input data. But it's hard to interpret many-to-many relationships. Humans are able to do this for all previous information stored in memory whenever an image, text or video appears.

## Discussion

### What's the difference between Information Retrieval (IR) and Information Extraction (IE)?

IR is about searching and retrieving unstructured documents that fulfil a specific search query. IE is about obtaining structured representation of key information from unstructured documents. For example, IR can help us find all documents about fishing whereas IE can tell who went fishing and what type of fish was caught. 

Information retrieval is the process of gathering information or sources that are appropriate to the topic from a collection of raw text data. 

Information extraction is a kind of automated process where rule-based algorithm is applied to structured data after it is obtained from any unstructured source. It involves NLP techniques of cleaning and arranging it in a matrix. It also involves activities like automatic annotation or content extraction. 


### Could you mention some applications of IR?

Web search is an example of IR. A search engine might search billions of web documents to respond to a query such as "Italian restaurants near me" or "flights to London tomorrow". A more personal example is to search through emails. IR in this case is typically executed by the mail program such as GMail. 

Searching through online discussion forums or Q&A archives is another IR application. Text classification is typically used to classify posts, questions and answers. 

In any enterprise system, we might need to retrieve patent documents, research papers or other publications based on a keyword or phrase. In a library, we might want to obtain books with specific words in its title, or by author name, or by genre. In an e-commerce site, we might want to apply various criteria and see a filtered list of products. 

As an example of IR for multimedia, MPEG-7 is a standard that's used to describe multimedia content. This metadata can be searched to retrieve audio/video clips that are of interest to the user. 


### What are some essential terms used in information retrieval?

While the term "record" is common in databases, the equivalent term in IR is **document**. It refers to a basic unit of data stored in the system. A document can be a newspaper article, an encyclopaedia entry, a paragraph in a report, a web page, an entire website, etc. It really depends on the application. 

A set of documents indexed and stored is called a **collection**. Alternative names include archive, corpus, and digital library. 

**Query** is what the user submits to the system to meet her information need. A query is composed of one or more terms. A **term** is a lexical item but it could be a phrase as well. 

To search through each document is a slow inefficient process. Instead, each document is indexed in advance. The **index** is a structured representation of the document. Searching through this representation is much faster. A similar representation of the query aids the retrieval function. 

In practice, IR systems use **inverted index** that maps each index term to a list of documents in which it occurs.  


### Does information retrieval search through structured or unstructured documents?

Information retrieval typically concerns itself with **unstructured documents**. IR started with a focus on text. More recently, due to machine learning and improved processing capability, other types such as image, audio, and video can also be retrieved. 

While documents may be unstructured, often they are accompanied by **structured metadata**. Metadata is data about the document. For example, an email will have fields date, from, to, subject, and body. This is useful structured information. Another example is a photograph taken with a digital camera. Useful metadata might include date, camera model, exposure setting, and picture dimensions. 

Sometimes documents are **semi-structured** in the sense that specific information are in "standard" locations that heuristic algorithms can identify. Text documents with sections and sub-sections are also semi-structured. XML documents can be seen as semi-structured that IR systems can exploit. 


### What are the various models of IR?

IR models can be categorized into Classic, Structured and Browsing models. Classic models include Boolean, Vector Space and Probabilistic models. 

Boolean primitives (AND, OR, NOT) can be used in queries according to user's information need. This requires some skills for any complex IR. Different vocabularies will result in problems of string matching. This is the most used approach. 

Vector space models use vectors to map documents, phrases or terms. Words/terms can be categorized to high or low resolving power weights according to vector dimension of document. This helps in finding out positive/negative or similar/opposite matches and retrieve information. 

Probability distribution models use different distribution types to find out similarity between documents. It can be one of two types: similarity-based and expected-utility-based. TF-IDF and HMM can be used in different frameworks to generate multiple words within the same model, making comparison easier. HMM gives better accuracy than traditional TF-IDF models. 

Two broad matching strategies are: 

    + **Literal Term**: Vector Space Model (VSM), Hidden Markov Model (HMM), Language Model (LM)
    + **Concept**: Latent Semantic Analysis (LSA), Probabilistic Latent Semantic Analysis (PLSA), Topical Mixture Model (TMM)

### How is information retrieval represented?

Information is any unstructured form of data which consists of insights of any particular domain. It needs to be converted to a format that can then be parsed/extracted into structured form. Information in text format can be represented as document representation or query representation.

The content of a document can be represented as a collection of terms such as terms, phrases or other units. Every term will be having their weights which gives its importance to that document. A proper term weighting is needed according to various aspects of words, phrases, names, statistical, linguistic and other means for better IR. 

Query representation is based on conventional approach to formulate a query based on a keyword or set of keywords. But in several cases there's no relevant context for user to select appropriate keywords. In that case, maximal frequent sequences is used to retrieve relevant documents from an input document. A **bag-of-words model** is used to identify keywords and extract similar documents. 


### Which are some basic techniques used in information retrieval?

Let's say a query has the term 'processed'. A strict match for this word will leave out documents that contain variants such as 'processing' and 'process'. **Stemming** or **lemmatization** solve this but they create their own problems.  

Often it doesn't make sense to index words such as 'to', 'be', 'for', etc. A **stop list** of these high-frequency words are often omitted for indexing. 

Suppose the user searches for 'high blood pressure'. The IR system may also search for 'hypertension' and 'hypertension, renal'. This is called **query expansion**. Query expansion usually involves pre-generating a suitable thesaurus using **term clustering**. 

When a query has multiple terms, some terms may be more important than others. This is done by **term weighting**. **TF-IDF** is a popular approach. 

To improve IR performance, **relevance feedback** can be used, typically in VSM. User is shown a small set of retrieved documents to get feedback on what's relevant and what's not. The IR system then refines the query to improve performance. 


### What are some useful resources on Information Retrieval?

Greengrass (2000) offers a comprehensive survey of IR. Nyamisa et al. (2017) is another survey paper. Zhou et al. (2006) introduces basic terms and concepts. There's also a comprehensive online IR glossary.

Stanford University's CS276 course on IR is worth studying. Among the books for IR, you can look into Manning et al. (2009), Croft et al. (2015) and Frakes et al. (1992).

For IR research, you'll need collections. Wiki Small/Large and CACM collections are available for free download. TIPSTER Complete is another useful collection. Other important collections include the Cranfield collection, TREC, GOV2, Reuters-21578, Reuters-RCV1, and 20 Newgroups. For cross-language IR, there's NTCIR and CLEF. 

To keep track of latest developments, ACM's Special Interest Group on Information Retrieval (SIGIR) is the place to visit.

## Milestones

Dec  
1931

Emanuel Goldberg receives a US patent for a machine that does information retrieval. The machine is opto-electric in nature. Light is passed through a negative search plate containing the search terms. Document matches are picked up by activating a photoelectric cell. All matches are recorded on a photographic plate. 

Mar  
1950

Calvin Mooers coins the term **Information Retrieval**. 

1952

H. P. Luhn describes an early IR system from IBM. It makes use of punched cards and a photoelectric scanning unit. Information is indexed and encoded into **record cards**. A query is input as a **search card**. Plug connections on a switchboard control how search terms are to be combined. This is therefore one of the earliest implementations of the **Boolean Model** for IR. 

1957

H. P. Luhn proposes a **statistical approach** to IR. Content can be identified based on the frequency of occurrence of words. This also means that documents can be indexed automatically based on the words they contain. This is therefore the first attempt at **automatic indexing**. The system uses statistical information to group words into "notional" families. 

1959

Cavin Mooers notes that often using information is not rewarded. Those who are at work are seen as getting the job done rather than fussing about information. Having information means you have to read it, understand it, store it carefully and make decisions based on it. These observations lead Mooers to define **Mooers' Law**,

> An information retrieval system will tend not to be used whenever it is more painful and troublesome for a customer to have information than for him not to have it.

1960

Maron and Kuhns publish a paper titled *On Relevance, Probabilistic Indexing and Information Retrieval*. Where previous IR systems returned matching documents, the idea in this paper is to rank documents by computing **probability of relevance**. The paper also introduces the notion of **query expansion**. In years to come, this goes on to become one of the most influential papers in the field of IR. 

1965

Salton and Lesk propose **SMART** retrieval system. It goes beyond Luhn's work of 1957. It uses the original search words along with their word stems. It uses synonym dictionaries to account for variations in vocabulary. It uses relations between words and surrounding words to determine the nature of content. Over the years, the SMART system develops or uses many important concepts such as vector model, term weighting, relevance feedback, and cosine similarity measure. 

1976

Robertson and Jones implement a **probabilistic model** to IR. From the distribution of index terms in documents, they determine term weights.  This work is perhaps the first real application of the probability to IR as proposed by Maron and Kuhns in 1960. 

1982

Salton et al. propose the **extended Boolean model**. Output from a traditional Boolean model is hard to control and retrieved documents are not ranked. The extended model assigns weights to terms in both queries and documents. This enables the model to rank retrieved documents based on similarity to queries. 

1988

Dumais et al. note two common problems in IR: same word has many meanings, different words express the same concept. These result in irrelevant documents or missed relevant documents respectively. They therefore propose a novel method that maps terms and documents into a lower dimensional **semantic space**. In later years, this field is named **Latent Semantic Analysis (LSA)**. 

1992

In the US, **Text Retrieval Conference (TREC)** is launched to facilitate research and collaboration in IR. It goes on to become an almost annual event. The growth of the web and the need for large-scale IR makes TREC all the more relevant. TREC produces test collections that are useful for IR research. 

Aug  
1998

At SIGIR'98, Ponte and Croft propose the use of **Language Modelling (LM)** to the task of IR. They note that typical IR systems lack a good indexing model. Existing indexing models make unwarranted parametric assumptions. They therefore propose a single non-parametric model for both indexing and retrieval. They show better performance compared to TF-IDF weighting. In 2001, Zhai and Lafferty study different smoothing techniques for such language models. 

Sep  
1998

Google is founded for web search. By now, many other search engines are already popular: Lycos (1994), Yahoo! Search (1995), Excite (1995), AltaVista (1995), Yandex (1997). These engines become the best instantiations of IR. They use features that until now were only experimental. 

2016

There's growing interest in applying neural networks to IR. This is called **Neural IR**. The network learns representations of queries and documents. In one application of Neural IR from 2019, two deep models are trained, one for image and one for text. This allows a user to retrieve images based on a textual query, or vice versa. However, early work on content-based image retrieval can be traced to the work of Wan et al. (2014).