Incorporating the Results of Co-word Analyses to Increase Search Variety for Information Retrieval

This research aims to incorporate the results of co-word analysis into information retrieval as a means to increase search variety for end users in the domain of information retrieval. Relevant data were first collected from Science Citation Index and Social Sciences Citation Index for the period 1987–1997. The results of co-word analysis on the data were compared with similar data obtained from three thesauri: Library and Information Science Abstracts thesaurus, Library of Congress Subject Headings and Thesaurus of Information Technology Terms. The differences detected between them indicate that the search variety may be increased by combining co-word analysis with the use of traditional thesauri. Subsequently, the results of coword analysis were compared with each other for two different periods (1987–1991 and 1992–1997). The changes among them were identified, implying that co-word analysis may be used directly to identify dynamic changes in its chosen domain area, thereby providing better up-to-date information to aid the information search process. Introduction The large amounts of information available through online databases, the Internet and other networks are changing the way we gather, process and retrieve information. However, gaining access to such information is often difficult, as a result of inconsistency involved in the processing of information and the way in which queries are expressed by searchers. Bates [2, 3] pointed out the gap between the end user and the indexer which guarantees the mismatches between user search terms and indexing terms on the same records. Although there are some ways to fill the gap, e.g. maintaining the consistency by enhancing the standards between indexers and the users or by developing user-friendly interfaces to link both the indexer and the users together, problems still exist in full-text searching, natural language searching, etc. Research also demonstrates that end users like to use a very wide range of different terms and none of those terms will occur very frequently [38]. Experienced online database searchers also know that if they conduct a thorough search, they need to use as many different terms and term variants as possible and they will scan various thesauri from the subject area and enter all the relevant terms that they are able to find [2, 3]. So, increasing search variety for the end users is an important aspect for success in information retrieval (IR) [2, 3]. A number of studies have been conducted, or are in the process of development, to increase the search variety for the end users [6, 9, 21, 33]. The commonality between bibliometrics and IR is not apparent. Bibliometrics deals with research products generated by researchers and scholars, while IR is principally concerned with information storage, search and retrieval. However, more holistic approaches to research in bibliometrics and IR would be likely to add to our understanding of research problems in both the 1 2 3 4 5 6 7 8 9 1110 1 2 3 4 5 6 7 8 9 20 1 2 113 4 5 6 7 8 9 30 1 2 3 4 5 6 7 8 9 40 1 2 3 4 5 6 7 8 9 50 1 2 Journal of Information Science, 26 (6) 2000, pp. 429–451 429 The effect of postings information on searching behaviour Incorporating the results of co-word analyses to increase search variety for information retrieval Correspondence to: Dr G.G. Chowdhury, Division of Information Studies, School of Applied Science, Nanyang Technological University, Nanyang Avenue, Singapore 639798. E-mail: asggchowdhury@ntu.edu. sg © 2000 Chartered Institute of Library and Information Professionals. All rights reserved. Not for commercial use or unauthorized distribution. at PENNSYLVANIA STATE UNIV on February 12, 2008 http://jis.sagepub. com Downloaded from fields [24]. Harter and Cheng [25] applied the co-citation concept in bibliometrics to IR and generated a new method or concept: the co-linked descriptor. Shalini [40] has utilised citation profiles to improve relevance in a two-stage retrieval system. Quoniam et al. [35] have employed Zipf’s law in IR to get a first impression of documents’ data sets by querying without any keywords. This paper proposes that the results of a co-word analysis may be used to generate search variety for the end users. The keyword sets generated through a coword analysis in the domain of IR, are compared with the corresponding keyword sets obtained from three thesauri: Library and Information Science Abstracts (LISA) thesaurus, Library of Congress Subject Headings (LCSH) and the Thesaurus of Information Technology Terms (TITT). The keyword clusters in the same subject domain generated during two different time-periods have been compared to identify the changes. Based on the findings, it is apparent that the results of a co-word analysis can produce keyword sets that are different from those that are obtained from traditional thesauri, and that the results of co-word analyses in the same subject domain produced different results over different time-periods. From this, it becomes evident that the results of co-word analyses may be used to yield better search variety in an IR environment.