OpenRaiser/Intern-Atlas huggingface dataset:
Tuning before feedback: combining ranking discovery and blind feedback for robust retrieval
Authors: Weiguo Fan, Ming Luo, Li Wang, Wensi Xi, Edward A. Fox
Venue: SIGIR
Year: 2004

Both ranking functions and user queries are very important factors affecting a search engine's performance. Prior research has looked at how to improve ad-hoc retrieval performance for existing queries while tuning the ranking function, or modify and expand user queries using a fixed ranking scheme using blind feedback. However, almost no research has looked at how to combine ranking function tuning and blind feedback together to improve ad-hoc retrieval performance. In this paper, we look at the performance improvement for ad-hoc retrieval from a more integrated point of view by combining the merits of both techniques. In particular, we argue that the ranking function should be tuned first, using user-provided queries, before applying the blind feedback technique. The intuition is that highly-tuned ranking offers more high quality documents at the top of the hit list, thus offers a stronger baseline for blind feedback. We verify this integrated model in a large scale heterogeneous collection and the experimental results show that combining ranking function tuning and blind feedback can improve search performance by almost 30% over the baseline Okapi system.
The patent retrieval task in the fourth NTCIR workshop
Authors: Atsushi Fujii, Makoto Iwayama, Noriko Kando
Venue: SIGIR
Year: 2004

This paper describes the Patent Retrieval Task in the Fourth NTCIR Workshop, and the test collections produced in this task. We perform the invalidity search task, in which each participant group searches a patent collection for the patents that can invalidate the demand in an existing claim. We also perform the automatic patent map generation task, in which the patents associated with a specific topic are organized in a multi-dimensional matrix.
Dependence language model for information retrieval
Authors: Jianfeng Gao, Jian-Yun Nie, Guangyuan Wu, Guihong Cao
Venue: SIGIR
Year: 2004

This paper presents a new dependence language modeling approach to information retrieval. The approach extends the basic language modeling approach based on unigram by relaxing the independence assumption. We integrate the linkage of a query as a hidden variable, which expresses the term dependencies within the query as an acyclic, planar, undirected graph. We then assume that a query is generated from a document in two stages: the linkage is generated first, and then each term is generated in turn depending on other related terms according to the linkage. We also present a smoothing method for model parameter estimation and an approach to learning the linkage of a sentence in an unsupervised manner. The new approach is compared to the classical probabilistic retrieval model and the previously proposed language models with and without taking into account term dependencies. Results show that our model achieves substantial and significant improvements on TREC collections.
Geotemporal querying of multilingual documents
Authors: Fredric C. Gey, Aitao Chen, Ray R. Larson, Kim Carl
Venue: SIGIR
Year: 2004

This demonstration utilizes a geographic information system interface to display multilingual news documents in time and space by extracting place names from text and matching them to a multilingual multi-script gazetteer which identifies the latitude and longitude of the location.
Sharing knowledge online (abstract only): a dream or reality?
Authors: Melanie Gnasa
Venue: SIGIR
Year: 2004

The Web provides a global platform for knowledge sharing. However, several shortcomings still arise from the absence of personalization and collaboration in Web searches. More effective retrieval techniques could be provided by means of transforming explicit knowledge into implicit knowledge. The approach presented in this paper is based on a peer-to-peer architecture and aims at complementing classical Web searches in terms of personalized ranking lists. These local rankings can be accumulated and evaluated in order to supplement the process of knowledge generation by building Virtual Knowledge Communities. Furthermore, the aggregation of ranking lists can be used to identify topics as well as communities of interest. Together with social aspects for community support, a framework for congenial Web search is defined.
Broken plural detection for arabic information retrieval
Authors: Abduelbaset Goweder, Massimo Poesio, Anne N. De Roeck
Venue: SIGIR
Year: 2004

Due to the high number of inflectional variations of Arabic words, empirical results suggest that stemming is essential for Arabic information retrieval. However, current light stemming algorithms do not extract the correct stem of irregular (so-called broken) plurals, which constitute ~10% of Arabic texts and ~41% of plurals. Although light stemming in particular has led to improvements in information retrieval [5, 6], the effects of broken plurals on the performance of information retrieval systems has not been examined.We propose a light stemmer that incorporates a broken plural recognition component, and evaluate it within the context of information retrieval. Our results show that identifying broken plurals and reducing them to their correct stems does result in a significant improvement in the performance of information retrieval systems.
Sentence completion
Authors: Korinna Grabski, Tobias Scheffer
Venue: SIGIR
Year: 2004

We discuss a retrieval model in which the task is to complete a sentence, given an initial fragment, and given an application specific document collection. This model is motivated by administrative and call center environments, in which users have to write documents with a certain repetitiveness. We formulate the problem setting and discuss appropriate performance metrics. We present an index-based retrieval algorithm and a cluster-based approach, and evaluate our algorithms using collections of emails that have been written by two distinct service centers.
Eye-tracking analysis of user behavior in WWW search
Authors: Laura A. Granka, Thorsten Joachims, Geri Gay
Venue: SIGIR
Year: 2004

We investigate how users interact with the results page of a WWW search engine using eye-tracking. The goal is to gain insight into how users browse the presented abstracts and how they select links for further exploration. Such understanding is valuable for improved interface design, as well as for more accurate interpretations of implicit feedback (e.g. clickthrough) for machine learning. The following presents initial results, focusing on the amount of time spent viewing the presented abstracts, the total number of abstract viewed, as well as measures of how thoroughly searchers evaluate their results set.
Comparison of using passages and documents for blind relevance feedback in information retrieval
Authors: Zhenmei Gu, Ming Luo
Venue: SIGIR
Year: 2004

This paper compares document blind feedback and passage blind feedback in Information Retrieval (IR), based on the work during the NRRC 2003 Reliable Information Access Summer workshop. The analysis of our experimental results shows overall consistency on the performance impact of using passages and documents for blind feedback. However, it is observed that the behavior of passage blind feedback, compared to document blind feedback, is both system dependent and topic dependent. The relationships between the performance impact of passage blind feedback and the number of feedback terms and the topic's average relevant document length, respectively, are examined to illustrate these dependencies.
Toward better weighting of anchors
Authors: David Hawking, Trystan Upstill, Nick Craswell
Venue: SIGIR
Year: 2004

Okapi BM25 scoring of anchor text surrogate documents has been shown to facilitate effective ranking in navigational search tasks over web data. We hypothesize that even better ranking can be achieved in certain important cases, particularly when anchor scores must be fused with content scores, by avoiding length normalisation and by reducing the attentuation of scores associated with high tf. Preliminary results are presented.
Locality preserving indexing for document representation
Authors: Xiaofei He, Deng Cai, Haifeng Liu, Wei-Ying Ma
Venue: SIGIR
Year: 2004

Document representation and indexing is a key problem for document analysis and processing, such as clustering, classification and retrieval. Conventionally, Latent Semantic Indexing (LSI) is considered effective in deriving such an indexing. LSI essentially detects the most representative features for document representation rather than the most discriminative features. Therefore, LSI might not be optimal in discriminating documents with different semantics. In this paper, a novel algorithm called Locality Preserving Indexing (LPI) is proposed for document indexing. Each document is represented by a vector with low dimensionality. In contrast to LSI which discovers the global structure of the document space, LPI discovers the local structure and obtains a compact document representation subspace that best detects the essential semantic structure. We compare the proposed LPI approach with LSI on two standard databases. Experimental results show that LPI provides better representation in the sense of semantic structure.
Query-related data extraction of hidden web documents
Authors: Yih-Ling Hedley, Muhammad Younas, Anne E. James, Mark Sanderson
Venue: SIGIR
Year: 2004

The larger amount of information on the Web is stored in document databases and is not indexed by general-purpose search engines (i.e., Google and Yahoo). Such information is\ndynamically generated through querying databases — which are\nreferred to as Hidden Web databases. Documents returned in\nresponse to a user query are typically presented using templategenerated Web pages. This paper proposes a novel approach that identifies Web page templates by analysing the textual contents and the adjacent tag structures of a document in order to extract query-related data. Preliminary results demonstrate that our approach effectively detects templates and retrieves data with high recall and precision.
Parsimonious language models for information retrieval
Authors: Djoerd Hiemstra, Stephen E. Robertson, Hugo Zaragoza
Venue: SIGIR
Year: 2004

We systematically investigate a new approach to estimating the parameters of language models for information retrieval, called parsimonious language models. Parsimonious language models explicitly address the relation between levels of language models that are typically used for smoothing. As such, they need fewer (non-zero) parameters to describe the data. We apply parsimonious models at three stages of the retrieval process: 1) at indexing time; 2) at search time; 3) at feedback time. Experimental results show that we are able to build models that are significantly smaller than standard models, but that still perform at least as well as the standard approaches.
The document as an ergodic markov chain
Authors: Eduard Hoenkamp, Dawei Song
Venue: SIGIR
Year: 2004

In recent years, statistical language models are being proposed as alternative to the vector space model. Viewing documents as language samples introduces the issue of defining a joint probability distribution over the terms. The present paper models a document as the result of a Markov process. It argues that this process is ergodic, which is theoretically plausible, and easy to verify in practice. The theoretical result is that the joint distribution can be easily obtained. This can also be applied for search resolutions other than the document level. We verified this in an experiment on query expansion demonstrating both the validity and the practicability of the method. This holds a promise for general language models.
Context-based methods for text categorisation
Authors: D. S. Hunnisett, W. J. Teahan
Venue: SIGIR
Year: 2004

We propose several context-based methods for text categorization. One method, a small modification to the PPM compression-based model which is known to significantly degrade compression performance, counter-intuitively has the opposite effect on categorization performance. Another method, called C-measure, simply counts the presence of higher order character contexts, and outperforms all other approaches investigated.
A study of methods for normalizing user ratings in collaborative filtering
Authors: Rong Jin, Luo Si
Venue: SIGIR
Year: 2004

The goal of collaborative filtering is to make recommendations for a test user by utilizing the rating information of users who share interests similar to the test user. Because ratings are determined not only by user interests but also the rating habits of users, it is important to normalize ratings of different users to the same scale. In this paper, we compare two different normalization strategies for user ratings, namely the Gaussian normalization method and the decoupling normalization method. Particularly, we incorporated these two rating normalization methods into two collaborative filtering algorithms, and evaluated their effectiveness on the EachMovie dataset. The experiment results have shown that the decoupling method for rating normalization is more effective than the Gaussian normalization method in improving the performance of collaborative filtering algorithms.
The overlap problem in content-oriented XML retrieval evaluation
Authors: Gabriella Kazai, Mounia Lalmas, Arjen P. de Vries
Venue: SIGIR
Year: 2004

Within the INitiative for the Evaluation of XML Retrieval(INEX) a number of metrics to evaluate the effectiveness of content-oriented XML retrieval approaches were developed. Although these metrics provide a solution towards addressing the problem of overlapping result elements, they do not consider the problem of overlapping reference components within the recall-base, thus leading to skewed effectiveness scores. We propose alternative metrics that aim to provide a solution to both overlap issues.
Refining term weights of documents using term dependencies
Authors: Hee-Soo Kim, Ikkyu Choi, Minkoo Kim
Venue: SIGIR
Year: 2004

When processing raw documents in Information Retrieval (IR) System, a term-weighting scheme is used to calculate the importance of each term which occurs in a document. However, most term-weighting schemes assume that a term is independent of the other terms. Term dependency is an indispensable consequence of language use [1]. Therefore, this assumption can make the information of a document being lost. In this paper, we propose new approach to refine term weights of documents using term dependencies discovered from a set of documents. Then, we evaluate our method with two experiments based on the vector space model [2] and the language model [3].
Information retrieval using word senses: root sense tagging approach
Authors: Sang-Bum Kim, Hee-Cheol Seo, Hae-Chang Rim
Venue: SIGIR
Year: 2004

Information retrieval using word senses is emerging as a good research challenge on semantic information retrieval. In this paper, we propose a new method using word senses in information retrieval: root sense tagging method. This method assigns coarse-grained word senses defined in WordNet to query terms and document terms by unsupervised way using co-occurrence information constructed automatically. Our sense tagger is crude, but performs consistent disambiguation by considering only the single most informative word as evidence to disambiguate the target word. We also allow multiple-sense assignment to alleviate the problem caused by incorrect disambiguation.Experimental results on a large-scale TREC collection show that our approach to improve retrieval effectiveness is successful, while most of the previous work failed to improve performances even on small text collection. Our method also shows promising results when is combined with pseudo relevance feedback and state-of-the-art retrieval function such as BM25.
Polynomial filtering in latent semantic indexing for information retrieval
Authors: Effrosini Kokiopoulou, Yousef Saad
Venue: SIGIR
Year: 2004

Latent Semantic Indexing (LSI) is a well established and effective framework for conceptual information retrieval. In traditional implementations of LSI the semantic structure of the collection is projected into the k-dimensional space derived from a rank-k approximation of the original term-by-document matrix. This paper discusses a new way to implement the LSI methodology, based on polynomial filtering. The new framework does not rely on any matrix decomposition and therefore its computational cost and storage requirements are low relative to traditional implementations of LSI. Additionally, it can be used as an effective information filtering technique when updating LSI models based on user feedback.
Corpus structure, language models, and ad hoc information retrieval
Authors: Oren Kurland, Lillian Lee
Venue: SIGIR
Year: 2004

Most previous work on the recently developed language-modeling approach to information retrieval focuses on document-specific characteristics, and therefore does not take into account the structure of the surrounding corpus. We propose a novel algorithmic framework in which information provided by document-based language models is enhanced by the incorporation of information drawn from clusters of similar documents. Using this framework, we develop a suite of new algorithms. Even the simplest typically outperforms the standard language-modeling approach in precision and recall, and our new interpolation algorithm posts statistically significant improvements for both metrics over all three corpora tested.
Belief revision for adaptive information retrieval
Authors: Raymond Y. K. Lau, Peter Bruza, Dawei Song
Venue: SIGIR
Year: 2004

Applying Belief Revision logic to model adaptive information retrieval is appealing since it provides a rigorous theoretical foundation to model partiality and uncertainty inherent in any information retrieval (IR) processes. In particular, a retrieval context can be formalised as a belief set and the formalised context is used to disambiguate vague user queries. Belief revision logic also provides a robust computational mechanism to revise an IR system's beliefs about the users' changing information needs. In addition, information flow is proposed as a text mining method to automatically acquire the initial IR contexts. The advantage of a belief-based IRsystem is that its IR behaviour is more predictable and explanatory. However, computational efficiency is often a concern when the belief revision formalisms are applied to large real-life applications. This paper describes our belief-based adaptive IR system which is underpinned by an efficient belief revision mechanism. Our initial experiments show that the belief-based symbolic IR model is more effective than a classical quantitative IR model. To our best knowledge, this is the first successful empirical evaluation of a logic-based IR model based on large IR benchmark collections.
Toponym resolution in text (abstract only): "which sheffield is it?"
Authors: Jochen L. Leidner
Venue: SIGIR
Year: 2004

Named entity tagging comprises the sub-tasks of identifying a text span and classifying it, but this view ignores the relationship between the entities and the world. Spatial and temporal entities ground events in space-time, and this relationship is vital for applications such as question answering and event tracking. There is much recent work regarding the temporal dimension (Setzer and Gaizauskas 2002, Mani and Wilson 2000), but no detailed study of the spatial dimension.I propose to investigate how spatial named entities (which are often referentially ambiguous) can be automatically resolved with respect to an extensional coordinate model (toponym resolution). To this end, various information sources including linguistic cue patterns, co-occurrence information, discourse/positional information, world knowledge (such as size and population) as well as minimality heuristics (Leidner et al. 2003) will be combined in a supervised machine learning regime.The major contributions of this research project will be a corpus of text manually annotated for spatial named entities with their model correlates as a training and evaluation resource, a novel method to spatially ground toponyms in text and a component-based evaluation based on this new reference corpus.
A music recommender based on audio features
Authors: Qing Li, Byeong Man Kim, Donghai Guan, Duk whan Oh
Venue: SIGIR
Year: 2004

Many collaborative music recommender systems (CMRS) have succeeded in capturing the similarity among users or items based on ratings, however they have rarely considered about the available information from the multimedia such as genres, let alone audio features from the media stream. Such information is valuable and can be used to solve several problems in RS. In this paper, we design a CMRS based on audio features of the multimedia stream. In the CMRS, we provide recommendation service by our proposed method where a clustering technique is used to integrate the audio features of music into the collaborative filtering (CF) framework in hopes of achieving better performance. Experiments are carried out to demonstrate that our approach is feasible.
Context-based question-answering evaluation
Authors: Elizabeth D. Liddy, Anne Diekema, Özgür Yilmazel
Venue: SIGIR
Year: 2004

In this poster, we will present the results of efforts we have undertaken to conduct evaluations of a QA system in a real world environment and to understand the nature of the dimensions on which users evaluate QA systems when given full reign to comment on whatever dimensions they deem important.
Supporting federated information sharing communities (abstract only)
Authors: Bicheng Liu
Venue: SIGIR
Year: 2004

Increasingly, the World Wide Web is being viewed as a means of creating web communities rather than simply as a means of publishing and delivering documents and services. In this research we develop the concept of Federated Information Sharing Communities (FISC), and associated architecture, that enables community-centred information systems to be constructed. Such systems provide a way for organisations, distributed workgroups and individuals to build up a federated community based on their common interests over the World Wide Web. To support communities, we develop capabilities that go beyond the generic retrieval of documents to include the ability to retrieve people, their interests and inter-relationships. We focus on providing social awareness "in the large" to help users understand the members within a community and the relationships between them: who is working on what topic, and who is working with whom. Within the FISC framework, we provide a viewpoint retrieval service to enable a user to construct visual contextual views of the community from the perspective of any community member. To evaluate these ideas we develop test beds to compare individual component technologies such as user and group profile construction and similarity matching, and we develop prototypes (Web Network and "CiteSeer Community") to explore the broader architecture and usage issues.
Cluster-based retrieval using language models
Authors: Xiaoyong Liu, W. Bruce Croft
Venue: SIGIR
Year: 2004

Previous research on cluster-based retrieval has been inconclusive as to whether it does bring improved retrieval effectiveness over document-based retrieval. Recent developments in the language modeling approach to IR have motivated us to re-examine this problem within this new retrieval framework. We propose two new models for cluster-based retrieval and evaluate them on several TREC collections. We show that cluster-based retrieval can perform consistently across collections of realistic size, and significant improvements over document-based retrieval can be obtained in a fully automatic manner and without relevance information provided by human.