OpenRaiser/Intern-Atlas huggingface dataset:
Search results using timeline visualizations
Authors: Omar Alonso, Michael Gertz, Ricardo A. Baeza-Yates
Venue: SIGIR
Year: 2007

1. OVERVIEW As search applications keep gathering new and diverse information sources, presenting relevant information anchored in time becomes more important. Time can help in describing the context of a document or document collection or in recreating a particular historical period. Temporal information is available in every document either explicitly or implicitly. Recognizing such temporal information and exploiting it for document retrieval and presentation purposes are important features that can significantly improve the functionality of today’s search applications. A look at functionality of any of the current search engines shows that temporal aspects of documents are exclusively used to sort the hit list by date, which is primarily the date a Web page or a document has been created or last modified. In this demonstration, we present a new approach in which search results are arranged in a well-defined timeline using a few existing visualization metaphors. To take advantage of time related information for IR purposes, we process each document in a document annotation pipeline that extracts temporal expressions and produces output in TimeML. Recently TimeML has emerged as a standard markup language for events and temporal expressions in natural language [1]. As the basis for anchoring documents in time, we assume a discrete representation of time based on the Gregorian Calendar, with a single day being an atomic time interval called chronon. Our base
Large-scale cluster-based retrieval experiments on Turkish texts
Authors: Ismail Sengör Altingövde, Rifat Ozcan, Huseyin Cagdas Öcalan, Fazli Can, Özgür Ulusoy
Venue: SIGIR
Year: 2007

We present cluster-based retrieval (CBR) experiments on the largest available Turkish document collection. Our experiments evaluate retrieval effectiveness and efficiency on both an automatically generated clustering structure and a manual classification of documents. In particular, we compare CBR effectiveness with full-text search (FS) and evaluate several implementation alternatives for CBR. Our findings reveal that CBR yields comparable effectiveness figures with FS. Furthermore, by using a specifically tailored cluster-skipping inverted index we significantly improve in-memory query processing efficiency of CBR in comparison to other traditional CBR techniques and even FS.
Using query contexts in information retrieval
Authors: Jing Bai, Jian-Yun Nie, Guihong Cao, Hugues Bouchard
Venue: SIGIR
Year: 2007

User query is an element that specifies an information need, but it is not the only one. Studies in literature have found many contextual factors that strongly influence the interpretation of a query. Recent studies have tried to consider the user's interests by creating a user profile. However, a single profile for a user may not be sufficient for a variety of queries of the user. In this study, we propose to use query-specific contexts instead of user-centric ones, including context around query and context within query. The former specifies the environment of a query such as the domain of interest, while the latter refers to context words within the query, which is particularly useful for the selection of relevant term relations. In this paper, both types of context are integrated in an IR model based on language modeling. Our experiments on several TREC collections show that each of the context factors brings significant improvements in retrieval effectiveness.
Finding similar experts
Authors: Krisztian Balog, Maarten de Rijke
Venue: SIGIR
Year: 2007

The task of finding people who are experts on a topic has recently received increased attention. We introduce a different expert finding task for which a small number of example experts is given (instead of a natural language query), and the system's task is to return similar experts. We define, compare, and evaluate a number of ways of representing experts, and investigate how the size of theinitial example set affects performance. We show that morefine-grained representations of candidates result in higher performance, and larger sample sets as input lead to improved precision.
ESTER: efficient search on text, entities, and relations
Authors: Hannah Bast, Alexandru Chitea, Fabian M. Suchanek, Ingmar Weber
Venue: SIGIR
Year: 2007

We present ESTER, a modular and highly efficient system for combined full-text and ontology search. ESTER builds on a query engine that supports two basic operations: prefix search and join. Both of these can be implemented very efficiently with a compact index, yet in combination provide powerful querying capabilities. We show how ESTER can answer basic SPARQL graph-pattern queries on the ontology by reducing them to a small number of these two basic operations. ESTER further supports a natural blend of such semantic queries with ordinary full-text queries. Moreover, the prefix search operation allows for a fully interactive and proactive user interface, which after every keystroke suggests to the user possible semantic interpretations of his or her query, and speculatively executes the most likely of these interpretations. As a proof of concept, we applied ESTER to the English Wikipedia, which contains about 3 million documents, combined with the recent YAGO ontology, which contains about 2.5 million facts. For a variety of complex queries, ESTER achieves worst-case query processing times of a fraction of a second, on a single machine, with an index size of about 4 GB.
Varying approaches to topical web query classification
Authors: Steven M. Beitzel, Eric C. Jensen, Abdur Chowdhury, Ophir Frieder
Venue: SIGIR
Year: 2007

Topical classification of web queries has drawn recent interest because of the promise it offers in improving retrieval effectiveness and efficiency. However, much of this promise depends on whether classification is performed before or after the query is used to retrieve documents. We examine two previously unaddressed issues in query classification: pre versus post-retrieval classification effectiveness and the effect of training explicitly from classified queries versus bridging a classifier trained using a document taxonomy. Bridging classifiers map the categories of a document taxonomy onto those of a query classification problem to provide sufficient training data. We find that training classifiers explicitly from manually classified queries outperforms the bridged classifier by 48% in F1 score. Also, a pre-retrieval classifier using only the query terms performs merely 11% worse than the bridged classifier which requires snippets from retrieved documents.
Boosting static pruning of inverted files
Authors: Roi Blanco, Alvaro Barreiro
Venue: SIGIR
Year: 2007

This paper revisits the static term-based pruning technique presented in Carmel et al., SIGIR 2001 for ad-hoc retrieval, addressing different issues concerning its algorithmic design not yet taken into account. Although the original technique is able to retain precision when a considerable part of the inverted file is removed, we show that it is possible to improve precision in some scenarios if some key design features are properly selected.
Random walk term weighting for information retrieval
Authors: Roi Blanco, Christina Lioma
Venue: SIGIR
Year: 2007

We present a way of estimating term weights for Information Retrieval (IR), using term co-occurrence as a measure of dependency between terms.We use the random walk graph-based ranking algorithm on a graph that encodes terms and co-occurrence dependencies in text, from which we derive term weights that represent a quantification of how a term contributes to its context. Evaluation on two TREC collections and 350 topics shows that the random walk-based term weights perform at least comparably to the traditional tf-idf term weighting, while they outperform it when the distance between co-occurring terms is between 6 and 30 terms.
Test theory for assessing IR test collections
Authors: David Bodoff, Pu Li
Venue: SIGIR
Year: 2007

How good is an IR test collection? A series of papers in recent years has addressed the question by empirically enumerating the consistency of performance comparisons using alternate subsets of the collection. In this paper we propose using Test Theory, which is based on analysis of variance and is specifically designed to assess test collections. Using the method, we not only can measure test reliability after the fact, but we can estimate the test collection's reliability before it is even built or used. We can also determine an optimal allocation of resources before the fact, e.g. whether to invest in more judges or queries. The method, which is in widespread use in the field of educational testing, complements data-driven approaches to assessing test collections. Whereas the data-driven method focuses on test results, test theory focuses on test designs. It offers unique practical results, as well as insights about the variety and implications of alternative test designs.
On the robustness of relevance measures with incomplete judgments
Authors: Tanuja Bompada, Chi-Chao Chang, John Chen, Ravi Kumar, Rajesh Shenoy
Venue: SIGIR
Year: 2007

We investigate the robustness of three widely used IR relevance measures for large data collections with incomplete judgments. The relevance measures we consider are the bpref measure introduced by Buckley and Voorhees [7], the inferred average precision (infAP) introduced by Aslam and Yilmaz [4], and the normalized discounted cumulative gain (NDCG) measure introduced by Järvelin and Kekäläinen [8]. Our main results show that NDCG consistently performs better than both bpref and infAP. The experiments are performed on standard TREC datasets, under different levels of incompleteness of judgments, and using two different evaluation methods, namely, the Kendall correlation measures order between system rankings and pairwise statistical significance testing; the latter may be of independent interest.
A flexible retrieval system of shapes in binary images
Authors: Gloria Bordogna, Luca Ghilardi, Simone Milesi, Marco Pagani
Venue: SIGIR
Year: 2007

This poster overviews the main characteristics of a flexible retrieval systems of shapes present in binary images and discusses some evaluation results. The system applies multiple indexing criteria of the shapes synthesizing distinct characteristics such as global features of the objects contour (Fourier Coefficients), boundary irregularities (Multifractal Spectrum), presence of concavities and convexities on the boundary (Contour Scale Space distribution). The system is flexible since it allows customizing the retrieval function to fit an application need. The query is a binary image containing the desired shape and a set of parameters specifying the distinct importance of the shape characteristics that must be taken into account to evaluate the relevance of the retrieved shapes. The retrieval function is then defined as a Flexible Multicriteria fusion Function producing ranked results. The evaluation experiments showed that this system can be suited to different retrieval purposes, and that generally the combination of the distinct shape indexing criteria increases both Recall and Precision with respect to the application of any single indexing criterion alone.
Lexical analysis for modeling web query reformulation
Authors: Alessandro Bozzon, Paul-Alexandru Chirita, Claudiu S. Firan, Wolfgang Nejdl
Venue: SIGIR
Year: 2007

Modeling Web query reformulation processes is still an unsolved problem. In this paper we argue that lexical analysis is highly beneficial for this purpose. We propose to use the variation in Query Clarity, as well as the Part-Of-Speech pattern transitions as indicators of user's search actions. Experiments with a log of 2.4 million queries showed our techniques to be more flexible than the current approaches, while also providing us with interesting insights into user's Web behavioral patterns.
Attention-based information retrieval
Authors: Georg Buscher
Venue: SIGIR
Year: 2007

In the proposed PhD thesis, it will be examined how attention data from the user, especially generated by an eye tracker, can be exploited in order to enhance and personalize information retrieval methods.
MRF based approach for sentence retrieval
Authors: Keke Cai, Chun Chen, Kangmiao Liu, Jiajun Bu, Peng Huang
Venue: SIGIR
Year: 2007

This poster focuses on the study of term context dependence in the application of sentence retrieval. Based on Markov Random Field (MRF), three forms of dependence among query terms are considered. Under different assumptions of term dependence relationship, three feature functions are defined, with the purpose to utilize association features between query terms in sentence to evaluate the relevance of sentence. Experimental results have proven the efficiency of the proposed retrieval models in improving the performance of sentence retrieval.
Robust test collections for retrieval evaluation
Authors: Ben Carterette
Venue: SIGIR
Year: 2007

Low-cost methods for acquiring relevance judgments can be a boon to researchers who need to evaluate new retrieval tasks or topics but do not have the resources to make thousands of judgments. While these judgments are very useful for a one-time evaluation, it is not clear that they can be trusted when re-used to evaluate new systems. In this work, we formally define what it means for judgments to be reusable: the confidence in an evaluation of new systems can be accurately assessed from an existing set of relevance judgments. We then present a method for augmenting a set of relevance judgments with relevance estimates that require no additional assessor effort. Using this method practically guarantees reusability: with as few as five judgments per topic taken from only two systems, we can reliably evaluate a larger set of ten systems. Even the smallest sets of judgments can be useful for evaluation of new systems.
Exploration of the tradeoff between effectiveness and efficiency for results merging in federated search
Authors: Suleyman Cetintas, Luo Si
Venue: SIGIR
Year: 2007

Federated search is the task of retrieving relevant documents from different information resources. One of the main research problems in federated search is to combine the results from different sources into a single ranked list. Recent work proposed a regression based method to download some documents from each ranked list of the different sources, calculated comparable scores for the documents and estimated mapping functions that transform source-specific scores into comparable scores. Experiments have shown that downloading more documents improves the accuracy of results merging. However downloading more documents increases the computation and communication costs.
Heavy-tailed distributions and multi-keyword queries
Authors: Surajit Chaudhuri, Kenneth Ward Church, Arnd Christian König, Liying Sui
Venue: SIGIR
Year: 2007

Intersecting inverted indexes is a fundamental operation for many applications in information retrieval and databases. Efficient indexing for this operation is known to be a hard problem for arbitrary data distributions. However, text corpora used in Information Retrieval applications often have convenient power-law constraints (also known as Zipf’s Law and long tails) that allow us to materialize carefully chosen combinations of multi-keyword indexes, which significantly improve worst-case performance without requiring excessive storage. These multi-keyword indexes limit the number of postings accessed when computing arbitrary index intersections. Our evaluation on an e-commerce collection of 20 million products shows that the indexes of up to four arbitrary keywords can be intersected while accessing less than 20 % of the postings in the largest single-keyword index.
A web page topic segmentation algorithm based on visual criteria and content layout
Authors: Idir Chibane, Bich-Liên Doan
Venue: SIGIR
Year: 2007

This paper presents experiments using an algorithm of web page topic segmentation that show significant precision improvement in the retrieval of documents issued from the Web track corpus of TREC 2001. Instead of processing the whole document, a web page is segmented into different semantic blocks according to visual criteria (such as horizontal lines, colors) and structural tags (such as headings <H1>~<H6>, paragraph <P>). We conclude that combining visual and content layout criteria gives the best results for increasing the precision: the ranking of the page is calculated for relevant segments of pages resulting from the segmentation algorithm.
The wild thing goes local
Authors: Kenneth Ward Church, Bo Thiesson
Venue: SIGIR
Year: 2007

Suppose you are on a mobile device with no keyboard (e.g., a cell phone) and you want to perform a "near me" search. Where is the nearest pizza? How do you enter queries quickly? T9? The Wild Thing encourages users to enter patterns with implicit and explicit wild cards (regular expressions). The search engine uses Microsoft Local Live logs to find the most likely queries for a particular location. For example, 7#6 is short-hand for the regular expression: /^[PQRS].*[ ][MNO].*/, which matches "post office" in many places (but "Space Needle" in Seattle). Some queries are more local than others. Pizza is likely everywhere, whereas "Boeing Company," is very likely in Seattle and Chicago, moderately likely nearby, and somewhat likely elsewhere. Smoothing is important. Not every query is observed everywhere.
The influence of caption features on clickthrough patterns in web search
Authors: Charles L. A. Clarke, Eugene Agichtein, Susan T. Dumais, Ryen W. White
Venue: SIGIR
Year: 2007

Web search engines present lists of captions, comprising title, snippet, and URL, to help users decide which search results to visit. Understanding the influence of features of these captions on Web search behavior may help validate algorithms and guidelines for their improved generation. In this paper we develop a methodology to use clickthrough logs from a commercial search engine to study user behavior when interacting with search result captions. The findings of our study suggest that relatively simple caption features such as the presence of all terms query terms, the readability of the snippet, and the length of the URL shown in the caption, can significantly influence users' Web search behavior.
Feature engineering for mobile (SMS) spam filtering
Authors: Gordon V. Cormack, José María Gómez Hidalgo, Enrique Puertas Sanz
Venue: SIGIR
Year: 2007

Mobile spam in an increasing threat that may be addressed using filtering systems like those employed against email spam. We believe that email filtering techniques require some adaptation to reach good levels of performance on SMS spam, especially regarding message representation. In order to test this assumption, we have performed experiments on SMS filtering using top performing email spam filters on mobile spam messages using a suitable feature representation, with results supporting our hypothesis.
Validity and power of t-test for comparing MAP and GMAP
Authors: Gordon V. Cormack, Thomas R. Lynam
Venue: SIGIR
Year: 2007

We examine the validity and power of the t-test, Wilcoxon test, and sign test in determining whether or not the difference in performance between two IR systems is significant. Empirical tests conducted on subsets of the TREC2004 Robust Retrieval collection indicate that the p-values computed by these tests for the difference in mean average precision (MAP) between two systems are very accurate fora wide range of sample sizes and significance estimates. Similarly, these tests have good power, with the t-test proving superior overall. The t-test is also valid for comparing geometric mean average precision (GMAP), exhibiting slightly superior accuracy and slightly inferior power than for MAPcomparison.
Power and bias of subset pooling strategies
Authors: Gordon V. Cormack, Thomas R. Lynam
Venue: SIGIR
Year: 2007

We define a method to estimate the random and systematic errors resulting from incomplete relevance assessments.Mean Average Precision (MAP) computed over a large number of topics with a shallow assessment pool substantially outperforms -- for the same adjudication effort MAP computed over fewer topics with deeper pools, and [email protected] computed with pools of the same depth. Move-to-front pooling,previously reported to yield substantially better rank correlation, yields similar power, and lower bias, compared tofixed-depth pooling.
Random walks on the click graph
Authors: Nick Craswell, Martin Szummer
Venue: SIGIR
Year: 2007

Search engines can record which documents were clicked for which query, and use these query-document pairs as "soft" relevance judgments. However, compared to the true judgments, click logs give noisy and sparse relevance information. We apply a Markov random walk model to a large click log, producing a probabilistic ranking of documents for a given query. A key advantage of the model is its ability to retrieve relevant documents that have not yet been clicked for that query and rank those effectively. We conduct experiments on click logs from image search, comparing our ("backward") random walk model to a different ("forward") random walk, varying parameters such as walk length and self-transition probability. The most effective combination is a long backward walk with high self-transition probability.
Query suggestion based on user landing pages
Authors: Silviu Cucerzan, Ryen W. White
Venue: SIGIR
Year: 2007

This poster investigates a novel query suggestion technique that selects query refinements through a combination of many users' post-query navigation patterns and the query logs of a large search engine. We compare this technique, which uses the queries that retrieve in the top-ranked search results places where searchers end up after post-query browsing (i.e., the landing pages), with an approach based on query refinements from user search sessions extracted from query logs. Our findings demonstrate the effectiveness of using landing pages for the direct generation of query suggestions, as well as the complementary nature of the suggestions it generates with regard to traditional query log based refinement methodologies.
A new approach for evaluating query expansion: query-document term mismatch
Authors: Tonya Custis, Khalid Al-Kofahi
Venue: SIGIR
Year: 2007

The effectiveness of information retrieval (IR) systems is influenced by the degree of term overlap between user queries and relevant documents. Query-document term mismatch, whether partial or total, is a fact that must be dealt with by IR systems. Query Expansion (QE) is one method for dealing with term mismatch. IR systems implementing query expansion are typically evaluated by executing each query twice, with and without query expansion, and then comparing the two result sets. While this measures an overall change in performance, it does not directly measure the effectiveness of IR systems in overcoming the inherent issue of term mismatch between the query and relevant documents, nor does it provide any insight into how such systems would behave in the presence of query-document term mismatch. In this paper, we propose a new approach for evaluating query expansion techniques. The proposed approach is attractive because it provides an estimate of system performance under varying degrees of query-document term mismatch, it makes use of readily available test collections, and it does not require any additional relevance judgments or any form of manual processing.
Mobile interface of the memoria project
Authors: Ricardo J. Dias, Rui Jesus, Rute Frias, Nuno Correia
Venue: SIGIR
Year: 2007

This project develops tools to manage personal memories that include a multimedia retrieval system and user interfaces for different devices. This paper and demonstration presents the mobile interface which allows browsing, retrieving, and taking pictures that are automatically annotated with GPS data and audio information. The multimedia retrieval system uses multimodal information: visual content, GPS metadata and audio information. The interface was evaluated in a cultural heritage site.