earch engines rock! Right? Without search engines, the Internet would be an enormous amount of disorganized information that would certainly be interesting but perhaps not very useful. Search engines help us in all kinds of tasks and are constantly improving result relevance. So, does even the tiniest scratch exist on this perfect image? We're afraid so. Contrary to popular belief, search engines don't answer questions. They merely provide fast access to the information people put on the Web. In fact, popular search engines, as opposed to query-answering systems, return Web pages matching the user's question rather than the question's answer. Luckily, this works most of the time, because people tend to place questions together with answers. The real problem arises when the information need expressed by the query is vague, too broad, or simply ill-defined. Then the range of covered topics becomes unmanageably large, confusing, and of doubtful value to the user. Search-results clustering aims to present information about the matching documents (see the "Related Work in Text Clustering" sidebar). It's like taking a step backward to grasp a bigger picture-we no longer care about individual documents, but about some underlying semantic structure capable of explaining why these documents constitute a good result to the query. To find this structure, we set a few goals: • identify groups of similar documents, • discover a textual description of the property making the documents similar, and • present these descriptions to the user in document clusters. Our approach reverses the traditional order of cluster discovery. Instead of calculating proximity between documents and then labeling the discovered groups, we first attempt to find good, conceptually varied cluster labels and then assign documents to the labels to form groups. We believe that only the commercial search engine Vivisimo (www.vivisimo com) uses a similar order of cluster discovery, but the details of that algorithm are unknown. According to the Collins English Dictionary, lingo is "a range of words or a style of language which is used in a particular situation or by a particular group of people." Each time a user issues a query on the Web, a new language is created, with its own characteristic vocabulary, phrases, and expressions. A successful Web-search-results clustering algorithm should speak its users' lingoes-that is, create thematic groups whose descriptions are easy to read and understand. Users will likely disregard groups with overly long or ambiguous descriptions, even though their content might be valuable. A Web search clustering algorithm must therefore aim to generate only clusters possessing meaningful, concise, and accurate labels. In conventional approaches, which determine group labels after discovering the actual cluster content, this task proves fairly difficult to accomplish. Numerical cluster representations might "know" that certain documents are similar, but they can't describe the actual relationship. In the Lingo description-comes-first approach, careful selection of label candidates is crucial. The algorithm must ensure that labels are significantly different while covering most of the topics in the input snippets. To find such candidates, we use the vector space model (VSM) and singular value decomposi-
Document clustering REF related to the grouping of the documents on the basis of their similarity.
A concept-driven algorithm for clustering search results
IEEE Intelligent Systems
IEEE Intelligent Systems
Computer Science