The Impact of Subject Indexes on Semantic Indeterminacy in Enterprise Document Retrieval

metadata in a medical collection concluded that full-text searching’s time has come. The researchers found that the documents returned by metadata search were more useful than full-text search, but they also found that full-text search 20 results could be improved to a level equivalent to those of the title and abstract searches by manipulating the ranking scheme in the search engine. This led the researchers to suggest that full-text searching’s time has come (Hemminger, Saelim, Sullivan, & Vision, 2007). However, it is important to note here that the searches performed were searches for information on specific genes that are usually represented in the searches by their acronyms. The literature reviewed thus far would suggest that their results and conclusion were based on an experiment that was biased in favor of full-text search. Acronyms do not exhibit the problems associated with Zipf’s rank-frequency distribution of words and orderly distribution of meanings. They will not take on different meanings as their frequency of appearance in a collection increases. A gene name acronym (e.g. – COMT) will most likely only have one meaning in a medical collection. Hemminger, et al.’s biased conclusion that full-text searching’s time has come, therefore, fails to discourage the proposed research. Using Subject Information to Improve Search Results It is safe to say that subject-based search has been around as long as libraries have been around. Even the library in ancient Alexandria appeared to have a form of subject-oriented cataloguing available to aid patrons in finding material (Witty, 1973). Drabenstott (2004) points out that domain experts tend to avoid using fulltext keyword based searches because they are too cumbersome to use, and that she and other experts in information retrieval tend to rely on subject, author, and bibliographic searches to find large sets of relevant documents. She calls for 21 research into means by which these domain experts’ tactics can be presented to end users of search tools (Drabenstott, 2004). If information retrieval researchers do not use keyword queries, why do we so often rely on them for everyone else? Zipf‘s Principle of Least Effort suggests this makes no sense (Zipf, 1949). If there are easier methods to be applied to the search for information, why do we continue to use the seemingly more difficult keyword search? The Library Science literature is littered with studies testing, in one way or another, the advantages to be gained by using subject metadata to improve search results. Studies of card catalog usage in the 1970s reported that 20-40% of catalog searches were subject searches (Lipetz, 1972; Swanson, 1972), while Markey, in a review of card catalog research expanded this range to 10-62% (Markey, 1984). Similar numbers were reported for subject searches as these catalogs moved online. A national study of online catalog users showed that subject or topical searches accounted for up to 59% of catalog searches. At the same time, the study reported that enhancements to such searches were desired by more than 40% of users (Matthews, Lawrence, & Ferguson, 1983). In a study of usage of the MELVYL online catalog system available to patrons of University of California libraries beginning in the 1990s, subject search numbers were found to be trending down (Larson, 1991). These findings led to many online catalogs being developed without the capability of searching the subject