OpenRaiser/Intern-Atlas huggingface dataset:
Using Oracle for natural language document retrieval an automatic query reformulation approach
Authors: Jens Grivolla
Venue: SIGIR
Year: 2005

In corporate applications, vast amounts of data are often stored in database systems such as Oracle. Apart from structured information this can include text documents which cannot easily be retrieved using traditional SQL queries.Oracle includes means to deal with full text document retrieval (called Oracle Text) that offer special query operators for searches inside text fields. We have explored the effect of these different operators for queries derived from natural language queries. This article compares the retrieval performances achieved with different automatic reformulations from natural language to Oracle SQL queries.
Topic themes for multi-document summarization
Authors: Sanda M. Harabagiu, V. Finley Lacatusu
Venue: SIGIR
Year: 2005

The problem of using topic representations for multi-document summarization (MDS) has received considerable attention recently. In this paper, we describe five different topic representations and introduce a novel representation of topics based on topic themes. We present eight different methods of generating MDS and evaluate each of these methods on a large set of topics used in past DUC workshops. Our evaluation results show a significant improvement in the quality of summaries based on topic themes over MDS methods that use other alternative topic representations.
Server selection methods in hybrid portal search
Authors: David Hawking, Paul Thomas
Venue: SIGIR
Year: 2005

The TREC.GOV collection makes a valuable web testbed for distributed information retrieval methods because it is naturally partitioned and includes 725 web-oriented queries with judged answers. It can usefully model aspects of government and large corporate portals. Analysis of the.gov data shows that a purely distributed approach would not be feasible for providing search on a.gov portal because of the large number (17,000+) of web sites and the high proportion that do not provide a search interface. An alternative hybrid approach, combining both distributed and centralized techniques, is proposed and server selection methods are evaluated within this framework using web-oriented evaluation methodology. A number of well-known algorithms are compared against representatives (highest anchor ranked page (HARP) and anchor weighted sum (AWSUM)) of a family of new selection methods which use link anchortext extracted from an auxiliary crawl to provide descriptions of sites which are not themselves crawled. Of the previously published methods, ReDDE substantially outperformed three variants of CORI and also outperformed a method based on Kullback-Leibler Divergence (extended) except on topic distillation. HARP and AWSUM performed best overall but were outperformed on the topic distillation task by extended KL Divergence.
Customizing information access according to domain and task knowledge: the ontoExplo system
Authors: Nathalie Hernandez, Josiane Mothe, Sandra Poulain
Venue: SIGIR
Year: 2005

In this paper we present a system that allows a user to explore or mine a document collection. This system is based on domain and task knowledge modelled in the form of ontologies and allows direct access both to information as it is stored and to information that is built from it. The system has been developed in Java.
Evaluating semantic indexing techniques through cross-language fingerprinting
Authors: Eduard Hoenkamp, Sander van Dijk
Venue: SIGIR
Year: 2005

Users in search of on-line document sources are usually looking for content, not words. Hence, IR researchers generally agree that search techniques should be geared toward the meaning underlying documents rather than toward the text itself. The most visible examples of such techniques are Latent Semantic Analysis (LSA), and the Hyperspace Analog to Language (HAL). If these techniques really uncover semantic dependencies, then they should be applicable across languages. We investigated this using electronic versions of three kinds of translated material: a novel, a popular treatise about cosmology, and a data base of technical specifications. We used the analogy of fingerprinting used in forensics to establish if individuals are related. Genetic fingerprinting uses enzymes to split the DNA and then compare the resulting band patterns. Likewise, in our research we use queries to split a document into fragments. If a search technique really isolates fragments related to the query, then a document and its translation should have similar band patterns. In this paper we (1) present the fingerprinting technique, (2) introduce the material used, and (3) report preliminary results of an evaluation for two semantic indexing techniques.
Live visual relevance feedback for query formulation
Authors: Eduard Hoenkamp, Gijs van Dinther
Venue: SIGIR
Year: 2005

Users browsing the Internet seem relatively satisfied with the performance of search engines. An optimistic explanation would be the high quality of search engines. A more pessimistic one would be that people just adapt easily to any new technology. A third explanation is people's ignorance about recall: as they simply don't know what relevant documents are missed, they can hardly be expected to worry about them. And so they easily conceive the result as the best they can get. To allow the user to better assess the quality of the search results, an algorithm was developed that computes a visual representation of the document space in the neighborhood of the user's query.The paper (1) outlines the algorithm, (2) shows how users can explore the neighborhood of a query, and (3) demonstrates how users can guess more judiciously whether they need to further elaborate their query to improve retrieval results.
A utility theoretic approach to determining optimal wait times in distributed information retrieval
Authors: Kartik Hosanagar
Venue: SIGIR
Year: 2005

Distributed IR systems query a large number of IR servers, merge the retrieved results and display them to users. Since different servers handle collections of different sizes, have different processing and bandwidth capacities, there can be considerable heterogeneity in their response times. The broker in the distributed IR system thus has to make decisions regarding terminating searches based on perceived value of waiting -- retrieving more documents -- and the costs imposed on users by waiting for more responses. In this paper, we apply utility theory to formulate the broker's decision problem. The problem is a stochastic nonlinear program. We use Monte Carlo simulations to demonstrate how the optimal wait time may be determined in the context of a comparison shopping engine that queries multiple store websites for price and product information. We use data gathered from 30 stores for a set of 60 books. Our research demonstrates how a broker can leverage information about past retrievals regarding distributions of server response time and relevance scores to optimize its performance. Our main contribution is the formulation of the decision model for optimal wait time and proposal of a solution method. Our results suggest that the optimal wait time is highly sensitive to the manner in which users value from a set of retrieved results differs from the sum of user value from each result evaluated independently. We also find that the optimal wait time increases with the size of the distributed collections, but only if user utility from a set of results is nearly equal to the sum of utilities from each result.
Boosted decision trees for word recognition in handwritten document retrieval
Authors: Nicholas R. Howe, Toni M. Rath, R. Manmatha
Venue: SIGIR
Year: 2005

Recognition and retrieval of historical handwritten material is an unsolved problem. We propose a novel approach to recognizing and retrieving handwritten manuscripts, based upon word image classification as a key step. Decision trees with normalized pixels as features form the basis of a highly accurate AdaBoost classifier, trained on a corpus of word images that have been resized and sampled at a pyramid of resolutions. To stem problems from the highly skewed distribution of class frequencies, word classes with very few training samples are augmented with stochastically altered versions of the originals. This increases recognition performance substantially. On a standard corpus of 20 pages of handwritten material from the George Washington collection the recognition performance shows a substantial improvement in performance over previous published results (75% vs 65%). Following word recognition, retrieval is done using a language model over the recognized words. Retrieval performance also shows substantially improved results over previously published results on this database. Recognition/retrieval results on a more challenging database of 100 pages from the George Washington collection are also presented.
Title extraction from bodies of HTML documents and its application to web page retrieval
Authors: Yunhua Hu, Guomao Xin, Ruihua Song, Guoping Hu, Shuming Shi, Yunbo Cao, Hang Li
Venue: SIGIR
Year: 2005

This paper is concerned with automatic extraction of titles from the bodies of HTML documents. Titles of HTML documents should be correctly defined in the title fields; however, in reality HTML titles are often bogus. It is desirable to conduct automatic extraction of titles from the bodies of HTML documents. This is an issue which does not seem to have been investigated previously. In this paper, we take a supervised machine learning approach to address the problem. We propose a specification on HTML titles. We utilize format information such as font size, position, and font weight as features in title extraction. Our method significantly outperforms the baseline method of using the lines in largest font size as title (20.9%-32.6% improvement in F1 score). As application, we consider web page retrieval. We use the TREC Web Track data for evaluation. We propose a new method for HTML documents retrieval using extracted titles. Experimental results indicate that the use of both extracted titles and title fields is almost always better than the use of title fields alone; the use of extracted titles is particularly helpful in the task of named page finding (23.1% -29.0% improvements).
A dual index model for contextual information retrieval
Authors: Xiangji Huang, Yan Rui Huang, Miao Wen
Venue: SIGIR
Year: 2005

In this paper, we propose a dual index model for contextual IR. For each query, we search against both document level and passage level indexes, and use the corresponding merge function to update the weights for both documents and paragraphs by combining the results from both indexes according to the granularity information in metadata. Experiments on 2004 TREC data show that a significant improvement can be made by using the dual index model.
Predicting query difficulty on the web by learning visual clues
Authors: Eric C. Jensen, Steven M. Beitzel, David A. Grossman, Ophir Frieder, Abdur Chowdhury
Venue: SIGIR
Year: 2005

We describe a method for predicting query difficulty in a precision-oriented web search task. Our approach uses visual features from retrieved surrogate document representations (titles, snippets, etc.) to predict retrieval effectiveness for a query. By training a supervised machine learning algorithm with manually evaluated queries, visual clues indicative of relevance are discovered. We show that this approach has a moderate correlation of 0.57 with precision at 10 scores from manual relevance judgments of the top ten documents retrieved by ten web search engines over 896 queries. Our findings indicate that difficulty predictors which have been successful in recall-oriented ad-hoc search, such as clarity metrics, are not nearly as correlated with engine performance in precision-oriented tasks such as this, yielding a maximum correlation of 0.3. Additionally, relying only on visual clues avoids the need for collection statistics that are required by these prior approaches. This enables our approach to be employed in environments where these statistics are unavailable or costly to retrieve, such as metasearch.
Study of cross lingual information retrieval using on-line translation systems
Authors: Rong Jin, Joyce Y. Chai
Venue: SIGIR
Year: 2005

Typical cross language retrieval requires special linguistic resources, such as bilingual dictionaries and parallel corpus. In this study, we focus on the cross lingual retrieval problem that only uses online translation systems. We compare two approaches: a translation-based approach that directly translates queries into the language of documents and then applies traditional information retrieval techniques; and a model-based approach that first learns a statistical translation model from the translations acquired from an online translation system and then applies the learned statistical model to cross lingual information retrieval. Our empirical study with ImageCLEF has shown the model-based approach performs significantly better than the translation-based approach.
3D viewpoint-based photo search and information browsing
Authors: Rieko Kadobayashi, Katsumi Tanaka
Venue: SIGIR
Year: 2005

We propose a new photo search method that uses three-dimensional (3D) viewpoints as queries. 3D viewpoint-based image retrieval is especially useful for searching collections of archaeological photographs,which contain many different images of the same object. Our method is designed to enable users to retrieve images that contain the same object but show a different view, and to browse groups of images taken from a similar viewpoint. We also propose using 3D scenes to query by example, which means that users do not have the problem of trying to formulate appropriate queries. This combination gives users an easy way of accessing not only photographs but also archived information.
Examination and enhancement of a ring-structured graphical search interface based on usability testing
Authors: Tomoko Kajiyama, Noriko Kando, Shin&apos, ichi Satoh
Venue: SIGIR
Year: 2005

We evaluated the interactive retrieval functionality of the Concentric Ring View according to a series of usability studies. This is a ring structure-based graphical user interface, like a planisphere, for image retrieval with multi-faceted metadata. Attribute values for each facet are arranged on a ring, and retrieved images are displayed inside using search keys derived from the attribute values on the bottom part of the rings. By rotating the rings, users can browse retrieved images while adjusting search keys. The first usability test conducted with thirty six participants confirmed that: (i) novice users, even junior high school students, could use this interface; (ii) users could find images better than anticipated; and (iii) the interface was good at choosing the first relevant image, but users could not refine retrieval because they were unable to reuse retrieved results. To solve this problem, we added two functionalities, personal history for reuse and relevance feedback. With these improvements, we named the new version of the interface Concentric Ring View F+. A second usability test with seven participants confirmed the effectiveness of this newer interface.
The loquacious user: a document-independent source of terms for query expansion
Authors: Diane Kelly, Vijay Deepak Dollu, Xin Fu
Venue: SIGIR
Year: 2005

In this paper we investigate the effectiveness of a document-independent technique for eliciting feedback from users about their information problems. We propose that such a technique can be used to elicit terms from users for use in query expansion and as a follow-up when ambiguous queries are initially posed by users. We design a feedback form to obtain additional information from users, administer the form to users after initial querying, and create a series of experimental runs based on the information that we obtained from the form. Results demonstrate that the form was successful at eliciting more information from users and that this additional information significantly improved retrieval performance. Our results further demonstrate a strong relationship between query length and performance.
Short comings of latent models in supervised settings
Authors: Vijay Krishnan
Venue: SIGIR
Year: 2005

The Aspect Model [1, 2] and the Latent Dirichlet Allocation Model [3, 4] are latent generative models proposed with the objective of modeling discrete data such as text. Though it is not explicitly published (to the best of our knowledge), it is reasonably well known in there search community that the Aspect Model does not perform very well in supervised settings and also that latent models are frequently not identifiable, i.e. their optimal parameters are not unique.In this paper, we make a much stronger claim about the pitfalls of commonly-used latent models. By constructing a small, synthetic, but by no means unrealistic corpus, we show that latent models have inherent limitations that prevent them from recovering semantically meaningful parameters from data generated from a reasonable generative distribution. In fact, our experiments with supervised classification using the Aspect Model, showed that its performance was rather poor, even worse than Naive Bayes, leading us to the synthetic study.We also analyze the scenario of using tempered EM and show that it would not plug the above shortcomings. Our analysis suggests that there is also some scope for improvement in the Latent Dirichlet Allocation Model(LDA) [3, 4]. We then use our insight into the shortcomings of these models, to come up with a promising variant of the LDA, that does not suffer from the aforesaid drawbacks. This could potentially lead to much better performance and model fit, in the supervised scenario.
A wireless natural language search engine
Authors: Jochen L. Leidner
Venue: SIGIR
Year: 2005

Web search using stationary (desktop) computers has become a pervasive activity. The mobile user in need of information, however, faces several problems in his or her quest to satisfy an information need. Mobile devices have small displays, and mobile user interfaces are often less then usable, because they impose the desktop Web search paradigm on the mobile user. We present a wireless search engine based on natural language queries transmitted via popular Small Message Service (SMS) text messages. Besides traditional keyword based queries, the system can accept questions or phrases and returns responses that contain likely answers (Figure 1) instead of traditional lists of hyperlinks. The additional precision gained from performing a linguistic analysis of the query helps extracting answers from Web pages directly, which requires no navigation. The system is implemented using a NLIR system residing on a server, which can translate questions or phrases into search engine queries or queries to SOAP Web services, where a gateway mediates between the mobile network and the Internet (Figure 2). Whereas on the desktop keyboard-based search still prevails, we find that in a mobile context question answering techniques can help overcome the output constraints.
Profile-based event tracking
Authors: Baoli Li, Wenjie Li, Qin Lu, Mingli Wu
Venue: SIGIR
Year: 2005

In this research, we focus on tracking topics that originate and evolve from a specific event. Intuitively, a few key elements of a target event, such as date, location, and persons involved, would be enough for making a decision on whether a test story is on-topic. Consequently, a profile-based event tracking method is proposed. We attempt to build an event profile from the given on-topic stories by robust information retrieval technologies. A feature selection metric and a recognized event clause are utilized to determine most (if not all) key semantic elements of the target event. Preliminary experiments on the TDT2 mandarin corpus show that this profile-based event tracking method is promising.
Evaluation of resources for question answering evaluation
Authors: Jimmy Lin
Venue: SIGIR
Year: 2005

Controlled and reproducible laboratory experiments, enabled by reusable test collections, represent a well-established methodology in modern information retrieval research. In order to confidently draw conclusions about the performance of different retrieval methods using test collections, their reliability and trustworthiness must first be established. Although such studies have been performed for ad hoc test collections, currently available resources for evaluating question answering systems have not been similarly analyzed. This study evaluates the quality of answer patterns and lists of relevant documents currently employed in automatic question answering evaluation, and concludes that they are not suitable for post-hoc experimentation. These resources, created from runs submitted by TREC QA track participants, do not produce fair and reliable assessments of systems that did not participate in the original evaluations. Potential solutions for addressing this evaluation gap and their shortcomings are discussed.
Assessing the term independence assumption in blind relevance feedback
Authors: Jimmy Lin, G. Craig Murray
Venue: SIGIR
Year: 2005

When applying blind relevance feedback for ad hoc document retrieval, is it possible to identify, a priori, the set of query terms that will most improve retrieval performance? Can this complex problem be reduced into the simpler one of making independent decisions about the performance effects of each query term? Our experiments suggest that, for the selection of terms for blind relevance feedback, the term independence assumption may be empirically justified.
Information sharing through rational links and viewpoint retrieval
Authors: Bicheng Liu, David J. Harper, Stuart N. K. Watt
Venue: SIGIR
Year: 2005

In this paper we present the concept of Federated Information Sharing Communities (FISC), which leverages organisational and social relationships with document content to provide community-centred information sharing and communication environments. Prominence is given to capabilities that go beyond the generic retrieval of documents to include the ability to retrieve people, their interests and inter-relationships. We focus on providing social awareness "in the large" to help users understand the members within community and the relationships between them. Within the FISC framework, we provide viewpoint retrieval to enable a user to construct member-specific view(s) of the community, based on their various topic interests. As proof of concept, we present the first FISC prototype based on the twenty-five year SIGIR collection and examples of operational results.
A maximum coherence model for dictionary-based cross-language information retrieval
Authors: Yi Liu, Rong Jin, Joyce Y. Chai
Venue: SIGIR
Year: 2005

One key to cross-language information retrieval is how to efficiently resolve the translation ambiguity of queries given their short length. This problem is even more challenging when only bilingual dictionaries are available, which is the focus of this paper. In the previous research of cross-language information retrieval using bilingual dictionaries, the word co-occurrence statistics is used to determine the most likely translations of queries. In this paper, we propose a novel statistical model, named ``maximum coherence model'', which estimates the translation probabilities of query words that are consistent with the word co-occurrence statistics. Unlike the previous work, where a binary decision is made for the selection of translations, the new model maintains the uncertainty in translating query words when their sense ambiguity is difficult to resolve. Furthermore, this new model is able to estimate translations of multiple query words simultaneously. This is in contrast to many previous approaches where translations of individual query words are determined independently. Empirical studies with TREC datasets have shown that the maximum coherence model achieves a relative 10% - 40% improvement in cross-language information retrieval, comparing to other approaches that also use word co-occurrence statistics for sense disambiguation.