OpenRaiser/Intern-Atlas huggingface dataset:
Context sensitive vocabulary and its application in protein secondary structure prediction
Authors: Yan Liu, Jaime G. Carbonell, Judith Klein-Seetharaman, Vanathi Gopalakrishnan
Venue: SIGIR
Year: 2004

Protein secondary structure prediction is an important step towards understanding the relation between protein sequence and structure. However, most current prediction methods use features difficult for biologists to interpret. In this paper, we present a new method that applies information retrieval techniques to solve the problem:we extract a context sensitive biological vocabulary for protein sequences and apply text classification methods to predict protein secondary structure. Experimental results show that our method performs comparably to the state-of-art methods. Furthermore, the context sensitive vocabularies can serve as a useful tool to discover meaningful regular expression patterns for protein structures.
Supporting federated information sharing communities
Authors: Bicheng Liu, David J. Harper, Stuart N. K. Watt
Venue: SIGIR
Year: 2004

In this paper we describe the concept of Federated Information Sharing Communities (FISC), and associated architecture, which provide a way for organisations, distributed workgroups and individuals to build up a federated community based on their common interests over the World Wide Web. To support communities, we develop capabilities that go beyond the generic retrieval of documents to include the ability to retrieve people, their interests and inter-relationships. We focus on providing social awareness "in the large" to help users understand the members within a community and the relationships between them. Within the FISC framework, we provide viewpoint retrieval to enable a user to construct visual contextual views of the community from the perspective of any community member. To evaluate these ideas we develop test beds to compare individual component technologies such as user and group profile construction and similarity matching, and we develop prototypes to explore the broader architecture and usage issues.
Configurable indexing and ranking for XML information retrieval
Authors: Shaorong Liu, Qinghua Zou, Wesley W. Chu
Venue: SIGIR
Year: 2004

Indexing and ranking are two key factors for efficient and effective XML information retrieval. Inappropriate indexing may result in false negatives and false positives, and improper ranking may lead to low precisions. In this paper, we propose a configurable XML information retrieval system, in which users can configure appropriate index types for XML tags and text contents. Based on users' index configurations, the system transforms XML structures into a compact tree representation, Ctree, and indexes XML text contents. To support XML ranking, we propose the concepts of "weighted term frequency" and "inverted element frequency," where the weight of a term depends on its frequency and location within an XML element as well as its popularity among similar elements in an XML dataset. We evaluate the effectiveness of our system through extensive experiments on the INEX 03 dataset and 30 content and structure (CAS) topics. The experimental results reveal that our system has significantly high precision at low recall regions and achieves the highest average precision (0.3309) as compared with 38 official INEX 03 submissions using the strict evaluation metric.
A search engine for imaged documents in PDF files
Authors: Yue Lu, Li Zhang, Chew Lim Tan
Venue: SIGIR
Year: 2004

Proceedings of Sheffield SIGIR - Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval
Information extraction using two-phase pattern discovery
Authors: Liping Ma, John Shepherd
Venue: SIGIR
Year: 2004

This paper presents a new two-phase pattern (2PP) discovery technique for information extraction. 2PP consists of orthographic pattern discovery (OPD) and semantic pattern discovery (SPD) where the OPD determines the structural features from an identified region of a document and the SPD discovers a dominant semantic pattern for the region via inference, apposition and analogy. Then the discovered pattern is applied back into the region to extract required data items through pattern matching. We evaluated 2PP using 6500 data items and obtained effective result.
Reliability and verification of natural language text on the world wide web (abstract only)
Authors: Melanie J. Martin
Venue: SIGIR
Year: 2004

The hypothesis that information on the Web can be verified automatically, with minimal user interaction, will be tested by building and evaluating an interactive system. In this paper, verification is defined as a reasonable determination of the truth or correctness of a statement by examination, research, or comparison with similar text. The system will contain modules for reliability ranking, query processing, document retrieval, and document clustering based on agreement. The query processing and document retrieval components will use standard IR techniques. The reliability module will estimate the likelihood that a statement on the Web can be trusted using standards developed by information scientists, as well as linguistic aspects of the page and the link structure of associated web pages. The clustering module will cluster relevant documents based on whether or not they agree or disagree with the statement to be verified. Relevant references are discussed.
Triangulation without translation
Authors: James Mayfield, Paul McNamee
Venue: SIGIR
Year: 2004

Transitive retrieval and triangulation have been proposed as ways to improve cross-language retrieval quality when translation resources have poor lexical coverage. We demonstrate that cross-language retrieval is viable for European languages with no translation resources at all; that transitive retrieval without translation does not suffer the drop-off in retrieval quality sometimes reported for transitive retrieval with translation; and that triangulation that combines multiple transitive runs with no translation can boost performance over direct translation-free retrieval.
A collaborative filtering algorithm and evaluation metric that accurately model the user experience
Authors: Matthew R. McLaughlin, Jonathan L. Herlocker
Venue: SIGIR
Year: 2004

Collaborative Filtering (CF) systems have been researched for over a decade as a tool to deal with information overload. At the heart of these systems are the algorithms which generate the predictions and recommendations.In this article we empirically demonstrate that two of the most acclaimed CF recommendation algorithms have flaws that result in a dramatically unacceptable user experience.In response, we introduce a new Belief Distribution Algorithm that overcomes these flaws and provides substantially richer user modeling. The Belief Distribution Algorithm retains the qualities of nearest-neighbor algorithms which have performed well in the past, yet produces predictions of belief distributions across rating values rather than a point rating value.In addition, we illustrate how the exclusive use of the mean absolute error metric has concealed these flaws for so long, and we propose the use of a modified Precision metric for more accurately evaluating the user experience.
Feature selection using linear classifier weights: interaction with classification models
Authors: Dunja Mladenic, Janez Brank, Marko Grobelnik, Natasa Milic-Frayling
Venue: SIGIR
Year: 2004

This paper explores feature scoring and selection based on weights from linear classification models. It investigates how these methods combine with various learning models. Our comparative analysis includes three learning algorithms: Naïve Bayes, Perceptron, and Support Vector Machines (SVM) in combination with three feature weighting methods: Odds Ratio, Information Gain, and weights from linear models, the linear SVM and Perceptron. Experiments show that feature selection using weights from linear SVMs yields better classification performance than other feature weighting methods when combined with the three explored learning algorithms. The results support the conjecture that it is the sophistication of the feature weighting method rather than its apparent compatibility with the learning algorithm that improves classification performance.
Evaluation of the real and perceived value of automatic and interactive query expansion
Authors: Yael Nemeth, Bracha Shapira, Meirav Taieb-Maimon
Venue: SIGIR
Year: 2004

The paper describes a user study examining methods for improving users queries, specifically interactive and automatic query expansion and advanced search options. The user study includes subjective and objective evaluation of the effect of the above methods and a comparison between the real and perceived effect.
Understanding combination of evidence using generative probabilistic models for information retrieval (abstract only)
Authors: Paul Ogilvie
Venue: SIGIR
Year: 2004

Structured documents, rich information needs, and detailed information about users are becoming more pervasive within everyday computing usage. Applications such as Question Answering, reading tutors, and XML retrieval demand more robust retrieval on richly annotated documents. In order to effectively serve these applications, the community will need a better understanding of the combination of evidence. In this work, I propose that the use of simple generative probabilistic models will be an effective framework for these problems. Statistical language models, which are a special case of generative probabilistic models, have been used extensively within recent Information Retrieval research. Their flexibility has been very effective in adapting to numerous tasks and problems. I propose to extend the statistical language modeling framework to handle rich information needs and documents with structural and linguistic annotations. Much of the prior work on combination of evidence has had few well-studied theoretical contributions, so I also propose to develop a sounder theoretical basis which gives more predictable results.
Usefulness of hyperlink structure for query-biased topic distillation
Authors: Vassilis Plachouras, Iadh Ounis
Venue: SIGIR
Year: 2004

In this paper, we introduce an information theoretic method for estimating the usefulness of the hyperlink structure induced from the set of retrieved documents. We evaluate the effectiveness of this method in the context of an optimal Bayesian decision mechanism, which selects the most appropriate retrieval approaches on a per-query basis for two TREC tasks. The estimation of the hyperlink structure's usefulness is stable when we use different weighting schemes, or when we employ sampling of documents to reduce the computational overhead. Next, we evaluate the effectiveness of the hyperlink structure's usefulness in a realistic setting, by setting the thresholds of a decision mechanism automatically. Our results show that improvements over the baselines are obtained.
A search engine for historical manuscript images
Authors: Toni M. Rath, R. Manmatha, Victor Lavrenko
Venue: SIGIR
Year: 2004

Many museum and library archives are digitizing their large collections of handwritten historical manuscripts to enable public access to them. These collections are only available in image formats and require expensive manual annotation work for access to them. Current handwriting recognizers have word error rates in excess of 50% and therefore cannot be used for such material. We describe two statistical models for retrieval in large collections of handwritten manuscripts given a text query. Both use a set of transcribed page images to learn a joint probability distribution between features computed from word images and their transcriptions. The models can then be used to retrieve unlabeled images of handwritten documents given a text query. We show experiments with a training set of 100 transcribed pages and a test set of 987 handwritten page images from the George Washington collection. Experiments show that the precision at 20 documents is about 0.4 to 0.5 depending on the model. To the best of our knowledge, this is the first automatic retrieval system for historical manuscripts using text queries, without manual transcription of the original corpus.
Resource selection for domain-specific cross-lingual IR
Authors: Monica Rogati, Yiming Yang
Venue: SIGIR
Year: 2004

An under-explored question in cross-language information retrieval (CLIR) is to what degree the performance of CLIR methods depends on the availability of high-quality translation resources for particular domains. To address this issue, we evaluate several competitive CLIR methods - with different training corpora - on test documents in the medical domain. Our results show severe performance degradation when using a general-purpose training corpus or a commercial machine translation system (SYSTRAN), versus a domain-specific training corpus. A related unexplored question is whether we can improve CLIR performance by systematically analyzing training resources and optimally matching them to target collections. We start exploring this problem by suggesting a simple criterion for automatically matching training resources to target corpora. By using cosine similarity between training and target corpora as resource weights we obtained an average of 5.6% improvement over using all resources with no weights. The same metric yields 99.4% of the performance obtained when an oracle chooses the optimal resource every time.
Learning patterns to answer open domain questions on the web
Authors: Dmitri Roussinov, Jose Antonio Robles-Flores
Venue: SIGIR
Year: 2004

While being successful in providing keyword based access to web pages, commercial search portals still lack the ability to answer questions expressed in a natural language. We present a probabilistic approach to automated question answering on the Web, based on trainable patterns, answer triangulation and semantic filtering. In contrast to the other “shallow ” approaches, our approach is entirely self-learning. It does not require any manually created scoring and filtering rules while still performing comparably. It also performs better than other fully trainable approaches.
Forming test collections with no system pooling
Authors: Mark Sanderson, Hideo Joho
Venue: SIGIR
Year: 2004

Forming test collection relevance judgments from the pooled output of multiple retrieval systems has become the standard process for creating resources such as the TREC, CLEF, and NTCIR test collections. This paper presents a series of experiments examining three different ways of building test collections where no system pooling is used. First, a collection formation technique combining manual feedback and multiple systems is adapted to work with a single retrieval system. Second, an existing method based on pooling the output of multiple manual searches is re-examined: testing a wider range of searchers and retrieval systems than has been examined before. Third, a new approach is explored where the ranked output of a single automatic search on a single retrieval system is assessed for relevance: no pooling whatsoever. Using established techniques for evaluating the quality of relevance judgments, in all three cases, test collections are formed that are as good as TREC.
Image based gisting in CLIR
Authors: Mark Sanderson, Robert C. Pasley
Venue: SIGIR
Year: 2004

In this paper, we describe research which could lead to a novel approach to gathering an overview of a document in a foreign language. The research explores how much of the meaning of a document could be represented using images by researching the ability of subjects to derive the search term that might have been used to return a set of images from an image library. The Google image search engine was used to retrieve the images for this experiment, which uses English throughout. The results were analysed with respect to a previous paper [1] exploring ability to recognise concrete objects in hierarchies. It was found that there is a tendency to use one particular level of categorization.
Evaluating high accuracy retrieval techniques
Authors: Chirag Shah, W. Bruce Croft
Venue: SIGIR
Year: 2004

ABSTRACT Although information retrieval research has always been concernedwith improving the effectiveness of search, in some applications, such as information analysis, a more specific requirement exists forhigh accuracy retrieval. This means that achieving high precision in the top document ranks is paramount. In this paper we presentwork aimed at achieving high accuracy in ad-hoc document retrieval by incorporating approaches from question answering (QA).We focus on getting the first relevant result as high as possible in the ranked list and argue that traditional precision and recall are notappropriate measures for evaluating this task. We instead use the mean reciprocal rank (MRR) of the first relevant result. We eval-uate three different methods for modifying queries to achieve high accuracy. The experiments done on TREC data provide support forthe approach of using MRR and incorporating QA techniques for getting high accuracy in ad-hoc retrieval task. Categories and Subject Descriptors H.3.4 [Information Storage and Retrieval]: Systems and Soft-ware--Performance evaluation (efficiency and effectiveness) ; H.3.3 [Information Storage and Retrieval]: Information Search andRetrieval--Query formulation
Searching databases for sematically-related schemas
Authors: Gauri Shah, Tanveer Fathima Syeda-Mahmood
Venue: SIGIR
Year: 2004

In this paper, we address the problem of searching schema databases for semantically-related schemas. We first give a method of finding semantic similarity between pair-wise schemas based on tokenization, part-of-speech tagging, word expansion, and ontology matching. We then address the problem of indexing the schema database through a semantic hash table. Matching schemas in the database are found by hashing the query attributes and recording peaks in the histogram of schema hits. Results indicated a 90% improvement in search performance while maintaining high precision and recall.
Web-page classification through summarization
Authors: Dou Shen, Zheng Chen, Qiang Yang, Hua-Jun Zeng, Benyu Zhang, Yuchang Lu, Wei-Ying Ma
Venue: SIGIR
Year: 2004

Web-page classification is much more difficult than pure-text classification due to a large variety of noisy information embedded in Web pages. In this paper, we propose a new Web-page classification algorithm based on Web summarization for improving the accuracy. We first give empirical evidence that ideal Web-page summaries generated by human editors can indeed improve the performance of Web-page classification algorithms. We then propose a new Web summarization-based classification algorithm and evaluate it along with several other state-of-the-art text summarization algorithms on the LookSmart Web directory. Experimental results show that our proposed summarization-based classification algorithm achieves an approximately 8.8% improvement as compared to pure-text-based classification algorithm. We further introduce an ensemble classifier using the improved summarization algorithm and show that it achieves about 12.9% improvement over pure-text based methods.
Restrictive clustering and metaclustering for self-organizing document collections
Authors: Stefan Siersdorfer, Sergej Sizov
Venue: SIGIR
Year: 2004

This paper addresses the problem of automatically structuring heterogenous document collections by using clustering methods. In contrast to traditional clustering, we study restrictive methods and ensemble-based meta methods that may decide to leave out some documents rather than assigning them to inappropriate clusters with low confidence. These techniques result in higher cluster purity, better overall accuracy, and make unsupervised self-organization more robust. Our comprehensive experimental studies on three different real-world data collections demonstrate these benefits. The proposed methods seem particularly suitable for automatically substructuring personal email folders or personal Web directories that are populated by focused crawlers, and they can be combined with supervised classification techniques.
Multiple sources of evidence for XML retrieval
Authors: Börkur Sigurbjörnsson, Jaap Kamps, Maarten de Rijke
Venue: SIGIR
Year: 2004

Document-centric XML collections contain text-rich documents, marked up with XML tags. The tags add lightweight semantics to the text. Querying such collections calls for a hybrid query language: the text-rich nature of the documents suggest a content-oriented (IR) approach, while the mark-up allows users to add structural constraints to their IR queries. We will show how evidence for relevancy from different sources helps to answer such hybrid queries. We evaluate our methods using the INEX 2003 test set, and show that structural hints in hybrid queries help to improve retrieval effectiveness.
Assigning identifiers to documents to enhance the clustering property of fulltext indexes
Authors: Fabrizio Silvestri, Salvatore Orlando, Raffaele Perego
Venue: SIGIR
Year: 2004

Web Search Engines provide a large-scale text document retrieval service by processing huge Inverted File indexes. Inverted File indexes allow fast query resolution and good memory utilization since their d-gaps representation can be effectively and efficiently compressed by using variable length encoding methods. This paper proposes and evaluates some algorithms aimed to find an assignment of the document identifiers which minimizes the average values of d-gaps, thus enhancing the effectiveness of traditional compression methods. We ran several tests over the Google contest collection in order to validate the techniques proposed. The experiments demonstrated the scalability and effectiveness of our algorithms. Using the proposed algorithms, we were able to sensibly improve (up to 20.81%) the compression ratios of several encoding schemes.
On evaluating web search with very few relevant documents
Authors: Ian Soboroff
Venue: SIGIR
Year: 2004

Many common web searches by their nature have a very small number of relevant documents. Homepage and "namedpage" searching are known-item searches where there is only a single relevant document. Topic distillation is a special kind of topical relevance search where the user wishes to find a few key web sites rather than every relevant web page. Because these types of searches are so common, web search evaluations have come to focus on tasks where there are very few relevant documents. Evaluations with few relevant documents pose special challenges for current metrics. In particular, the TREC 2003 topic distillation evaluation is unable to distinguish most submitted runs from each other.
A session-based search engine
Authors: Smitha Sriram, Xuehua Shen, ChengXiang Zhai
Venue: SIGIR
Year: 2004

In this poster, we describe a novel session-based search engine, which puts the search in context. The search engine has a number of session-based features including expansion of the current query with user query history and clickthrough data (title and summary of clicked web pages) in the same search session and the session boundary recognition through temporal closeness and probabilistic similarity between query terms. In addition, the search engine visualizes the rank change of web pages as different queries are submitted in the same search session to help the user reformulate the query.