The Use of WordNet in Information Retrieval

WordNet has been used in information retrieval research by many researchers, but failed to improve the performance of their retrieval system. Thereby in this paper we investigate why the use of WordNet has not been successful. Based on this analysis we propose a method of making WordNet more useful in information retrieval applications. Experiments using several standard information retrieval test collections show that our method results in a significant improvement of information retrieval performance. 1 I n t r o d u c t i o n Development of WordNet began in 1985 at Princeton University (Miller, 1990). A team lead by Prof. George Miller aimed to create a source of lexical knowledge whose organization would reflect some of the recent findings of psycholinguistic research into the human lexicon. WordNet has been used in numerous natural language processing, such as part of speech tagging (Segond et al., 97), word sense disambiguation (Resnik, 1995), text categorization (Gomez-Hidalgo and Rodriguez, 1997), information extraction (Chai and Biermann, 1997), and so on with considerable success. However the usefulness of WordNet in information retrieval applications has been debatable. Information retrieval is concerned with locating documents relevant to a user's information needs from a collection of documents. The user describes his/her information needs with a query which consists of a number of words. The information retrieval system compares the query with documents in the collection and returns the documents that are likely to satisfy the user's information requirements. A fundamental weakness of current information retrieval methods is that the vocabulary that searchers use is often not the same as the one by which the information has been indexed. Query expansion is one method to solve this problem. The query is expanded using terms which have similar meaning or bear some relation to those in the query, increasing the chances of matching words in relevant documents. Expanded terms are generally taken from a thesaurus. Obviously, given a query, the information retrieval system must present all useful articles to the user. This objective is measured by recall, i.e. the proportion of relevant articles retrieved by the system. Conversely, the information retrieval system must not present any useless article to the user. This criteria is measured by precision, i.e. the proportion of retrieved articles that are relevant. Voorhees used WordNet as a tool for query expansion (Voorhees, 1994). She conducted experiments using the TREC collection (Voorhees and Harman, 1997) in which all terms in the queries were expanded using a combination of synonyms, hypernyms, and hyponyms. She set the weights of the words contained in the original query to 1, and used a combination of 0.1, 0.3, 0.5, 1, and 2 for the expansion terms. She then used the SMART Information Retrieval System Engine (Salton, 1971) to retrieve the documents. Through this method, Voorhees only succeeded in improving the performance on short queries and a tittle with no significant improvement for long queries. She further tried to use WordNet as a tool for word sense disambiguation (Voorhees, 1993) and applied it to text retrieval, but the performance of retrieval was degraded. Stairmand (Stairmand, 1997) used WordNet to compute lexical cohesion according to the method suggested by Morris (Morris and Hirst, 199 I), and applied this to information retrieval.