OpenRaiser/Intern-Atlas huggingface dataset:
Prefetching query results and its impact on search engines
Authors: Simon Jonassen, Berkant Barla Cambazoglu, Fabrizio Silvestri
Venue: SIGIR
Year: 2012

We investigate the impact of query result prefetching on the efficiency and effectiveness of web search engines. We propose offline and online strategies for selecting and ordering queries whose results are to be prefetched. The offline strategies rely on query log analysis and the queries are selected from the queries issued on the previous day. The online strategies select the queries from the result cache, relying on a machine learning model that estimates the arrival times of queries. We carefully evaluate the proposed prefetching techniques via simulation on a query log obtained from Yahoo! web search. We demonstrate that our strategies are able to improve various performance metrics, including the hit rate, query response time, result freshness, and query degradation rate, relative to a state-of-the-art baseline.
Inferring missing relevance judgments from crowd workers via probabilistic matrix factorization
Authors: Hyun Joon Jung, Matthew Lease
Venue: SIGIR
Year: 2012

In crowdsourced relevance judging, each crowd worker typically judges only a small number of examples, yielding a sparse and imbalanced set of judgments in which relatively few workers influence output consensus labels, particularly with simple consensus methods like majority voting. We show how probabilistic matrix factorization, a standard approach in collaborative filtering, can be used to infer missing worker judgments such that all workers influence output labels. Given complete worker judgments inferred by PMF, we evaluate impact in unsupervised and supervised scenarios. In the supervised case, we consider both weighted voting and worker selection strategies based on worker accuracy. Experiments on crowd judgments from the 2010 TREC Relevance Feedback Track show promise of the PMF approach merits further investigation and analysis.
Reactive index replication for distributed search engines
Authors: Flavio Paiva Junqueira, Vincent Leroy, Matthieu Morel
Venue: SIGIR
Year: 2012

Distributed search engines comprise multiple sites deployed across geographically distant regions, each site being specialized to serve the queries of local users. When a search site cannot accurately compute the results of a query, it must forward the query to other sites. This paper considers the problem of selecting the documents indexed by each site focusing on replication to increase the fraction of queries processed locally. We propose RIP, an algorithm for replicating documents and posting lists that is practical and has two important features. RIP evaluates user interests in an online fashion and uses only local data of a site. Being an online approach simplifies the operational complexity, while locality enables higher performance when processing queries and documents. The decision procedure, on top of being online and local, incorporates document popularity and user queries, which is critical when assuming a replication budget for each site. Having a replication budget reflects the hardware constraints of any given site. We evaluate RIP against the approach of replicating popular documents statically, and show that we achieve significant gains, while having the additional benefit of supporting incremental indexes.
Sketch-based image similarity search with a pen and paper interface
Authors: Ihab Al Kabary, Heiko Schuldt
Venue: SIGIR
Year: 2012

We present a novel and innovative user interface for query-by-sketching based image retrieval that exploits emergent interactive paper and digital pen technology. Users can draw sketches with a digital pen on interactive paper in a user-friendly way. The pen is able to capture the stroke vectors and to interactively stream them to the underlying content-based image retrieval (CBIR) system via the pen's Bluetooth interface. We present the integration of interactive paper/digital pen technology with QbS, our CBIR system tailored to Query-by-Sketching, and we demonstrate the use of the paper and pen interface together with QbS for three different collections: MIRFLICKR-25K, a cartoon collection, and a collection of medieval paper watermarks.
Content-based retrieval for heterogeneous domains: domain adaptation by relative aggregation points
Authors: Makoto P. Kato, Hiroaki Ohshima, Katsumi Tanaka
Venue: SIGIR
Year: 2012

We introduce the problem of domain adaptation for content-based retrieval and propose a domain adaptation method based on relative aggregation points (RAPs). Content-based retrieval including image retrieval and spoken document retrieval enables a user to input examples as a query, and retrieves relevant data based on the similarity to the examples. However, input examples and relevant data can be dissimilar, especially when domains from which the user selects examples and from which the system retrieves data are different. In content-based geographic object retrieval, for example, suppose that a user who lives in Beijing visits Kyoto, Japan, and wants to search for relatively inexpensive restaurants serving popular local dishes by means of a content-based retrieval system. Since such restaurants in Beijing and Kyoto are dissimilar due to the difference in the average cost and areas' popular dishes, it is difficult to find relevant restaurants in Kyoto based on examples selected in Beijing. We propose a solution for this problem by assuming that RAPs in different domains correspond, which may be dissimilar but play the same role. A RAP is defined as the expectation of instances in a domain that are classified into a certain class, e.g. the most expensive restaurant, average restaurant, and restaurant serving the most popular dishes. Our proposed method constructs a new feature space based on RAPs estimated in each domain and bridges the domain difference for improving content-based retrieval in heterogeneous domains. To verify the effectiveness of our proposed method, we evaluated various methods with a test collection developed for content-based geographic object retrieval. Experimental results show that our proposed method achieved significant improvements over baseline methods. Moreover, we observed that the search performance of content-based retrieval in heterogeneous domains was significantly lower than that in homogeneous domains. This finding suggests that relevant data for the same search intent depend on the search context, that is, the location where the user searches and the domain from which the system retrieves data.
Utilizing inter-document similarities in federated search
Authors: Savva Khalaman, Oren Kurland
Venue: SIGIR
Year: 2012

We demonstrate the merits of using inter-document similarities for federated search. Specifically, we study a results merging method that utilizes information induced from clusters of similar documents created across the lists retrieved from the collections. The method significantly outperforms state-of-the-art results merging approaches.
Gender-aware re-ranking
Authors: Eugene Kharitonov, Pavel Serdyukov
Venue: SIGIR
Year: 2012

In this paper we study usefulness of users' gender information for improving ranking of ambiguous queries in personalized and non-contextual settings. This study is performed as a sequence of offline re-ranking experiments and it demonstrates that the proposed gender-aware ranking features provide improvements in ranking quality. It is also shown that the proposed personalized features exhibit performance superior to non-contextual ones.
Building reputation and trust using federated search and opinion mining
Authors: Somayeh Khatiban
Venue: SIGIR
Year: 2012

The term online reputation addresses trust relationships amongst agents in dynamic open systems. These can appear as ratings, recommendations, referrals and feedback. Several reputation models and rating aggregation algorithms have been proposed. However, finding a trusted entity on the web is still an issue as all reputation systems work individually. The aim of this project is to introduce a global reputation system for electronic product reviews that aggregates people's opinions from different resources (e.g. e-commerce websites, and review) with the help federated search techniques and generate a high quality and trusted result. The first step is to choose a range of product review collections from e-commerce review systems (e.g. Amazon), online review sites (e.g. Epinions), social networks (e.g. Facebook), question and answering sites (e.g. Yahoo! Answers), and blog (e.g. My Nokia Blog) resources. By using a federated search approach the query (product name) will be broadcasted to the selected resources and the result will be a list of reputation data with various formats including star rating, text reviews, voting, video, and so on. The focus of this work is on review text data and star ratings.
Predicting lifespans of popular tweets in microblog
Authors: Shoubin Kong, Ling Feng, Gordon Guo-Zheng Sun, Kan Luo
Venue: SIGIR
Year: 2012

In microblog like Twitter, popular tweets are usually retweeted by many users. For different tweets, their lifespans (i.e., how long they will stay popular) vary. This paper presents a simple yet effective approach to predict the lifespans of popular tweets based on their static characteristics and dynamic retweeting patterns. For a potentially popular tweet, we generate a time series based on its first-hour retweeting information, and compare it with those of historic tweets of the same author and post time (at the granularity of hour). The top-k historic tweets are identified, whose mean lifespan is estimated as the lifespan of the new tweet. Our experiments on a three-month real data set from Tencent Microblog demonstrate the effectiveness of the approach.
Manhattan hashing for large-scale image retrieval
Authors: Weihao Kong, Wu-Jun Li, Minyi Guo
Venue: SIGIR
Year: 2012

Hashing is used to learn binary-code representation for data with expectation of preserving the neighborhood structure in the original feature space. Due to its fast query speed and reduced storage cost, hashing has been widely used for efficient nearest neighbor search in a large variety of applications like text and image retrieval. Most existing hashing methods adopt Hamming distance to measure the similarity (neighborhood) between points in the hashcode space. However, one problem with Hamming distance is that it may destroy the neighborhood structure in the original feature space, which violates the essential goal of hashing. In this paper, Manhattan hashing (MH), which is based on Manhattan distance, is proposed to solve the problem of Hamming distance based hashing. The basic idea of MH is to encode each projected dimension with multiple bits of natural binary code (NBC), based on which the Manhattan distance between points in the hashcode space is calculated for nearest neighbor search. MH can effectively preserve the neighborhood structure in the data to achieve the goal of hashing. To the best of our knowledge, this is the first work to adopt Manhattan distance with NBC for hashing. Experiments on several large-scale image data sets containing up to one million points show that our MH method can significantly outperform other state-of-the-art methods.
Queries without clicks: evaluating retrieval effectiveness based on user feedback
Authors: Athanasia Koumpouri, Vasiliki Simaki
Venue: SIGIR
Year: 2012

Until recently, the lack of user activity on search results was perceived as a sign of user dissatisfaction from retrieval performance. However, recent studies have reported that some queries might not be followed by clicks to the content of the retrieved results, because the search task can be satisfied in the list of retrieved results the user views without the need to click through them. In this paper, we propose a method for evaluating user satisfaction from the results of searches that are not followed by clickthrough activity to the retrieved results. We found that there is a strong association between some implicit measures of user activity and user's explicit satisfaction judgments. Moreover, we developed a predictive model of user satisfaction based on implicit measures, achieving accuracy up to 86%.
Fast on-line learning for multilingual categorization
Authors: Michelle Kovesi, Cyril Goutte, Massih-Reza Amini
Venue: SIGIR
Year: 2012

Multiview learning has been shown to be a natural and efficient framework for supervised or semi-supervised learning of multilingual document categorizers. The state-of-the-art co-regularization approach relies on alternate minimizations of a combination of language-specific categorization errors and a disagreement between the outputs of the monolingual text categorizers. This is typically solved by repeatedly training categorizers on each language with the appropriate regularizer. We extend and improve this approach by introducing an on-line learning scheme, where language-specific updates are interleaved in order to iteratively optimize the global cost in one pass. Our experimental results show that this produces similar performance as the batch approach, at a fraction of the computational cost.
Re-examining search result snippet examination time forrelevance estimation
Authors: Dmitry Lagun, Eugene Agichtein
Venue: SIGIR
Year: 2012

Previous studies of web search result examination have provided valuable insights in understanding and modelling searcher behavior. Yet, recent work (e.g., [3]) has been developed based on the assumption that the time a searcher spends examining a particular result abstract or snippet, correlates with result relevance. While this idea is intuitively attractive, to the best of our knowledge it has not been empirically tested. This poster investigates this hypothesis empirically, in a controlled setting, using eye tracking equipment to compare search result examination time with result relevance. Interestingly, while we replicate previous findings showing examination time to be indicative of whole-page relevance, we find that viewing time of individual results alone is a poor indicator of either absolute result relevance or even of pairwise preferences. Our results should not be taken as negating the usefulness of modeling searcher examination behavior, but rather to emphasize that snippet examination time is not in itself a good indicator of relevance.
Preliminary study of technical terminology for the retrieval of scientific book metadata records
Authors: Birger Larsen, Christina Lioma, Ingo Frommholz, Hinrich Schütze
Venue: SIGIR
Year: 2012

Books only represented by brief metadata (book records) are particularly hard to retrieve. One way of improving their retrieval is by extracting retrieval enhancing features from them. This work focusses on scientific (physics) book records. We ask if their technical terminology can be used as a retrieval enhancing feature. A study of 18,443 book records shows a strong correlation between their technical terminology and their likelihood of relevance. Using this finding for retrieval yields >+5% precision and recall gains.
Crowdsourcing for search evaluation and social-algorithmic search
Authors: Matthew Lease, Omar Alonso
Venue: SIGIR
Year: 2012

The first computers were people. Today, Internet-based access to 24/7 online human crowds has led to a renaissance of research in human computation and the advent of crowdsourcing. These new opportunities have brought a disruptive shift to research and practice for how we build intelligent systems today. Not only can labeled data for training and evaluation be collected faster, cheaper, and easier than ever before, but we now see human computation being integrated into the systems themselves, operating in concert with automation. This tutorial introduces opportunities and challenges of human computation and crowdsourcing, particularly for search evaluation and developing hybrid search solutions that integrate human computation with traditional forms of automated search. We review methodology and findings of recent research and survey current generation crowdsourcing platforms now available, analyzing methods, potential, and limitations across platforms.
Where is who: large-scale photo retrieval by facial attributes and canvas layout
Authors: Yu-Heng Lei, Yan-Ying Chen, Bor-Chun Chen, Lime Iida, Winston H. Hsu
Venue: SIGIR
Year: 2012

The ubiquitous availability of digital cameras has made it easier than ever to capture moments of life, especially the ones accompanied with friends and family. It is generally believed that most family photos are with faces that are sparsely tagged. Therefore, a better solution to manage and search in the tremendously growing personal or group photos is highly anticipated. In this paper, we propose a novel way to search for face photos by simultaneously considering attributes (e.g., gender, age, and race), positions, and sizes of the target faces. To better match the content and layout of the multiple faces in mind, our system allows the user to graphically specify the face positions and sizes on a query "canvas," where each attribute combination is defined as an icon for easier representation. As a secondary feature, the user can even place specific faces from the previous search results for appearance-based retrieval. The scenario has been realized on a tablet device with an intuitive touch interface. Experimenting with a large-scale Flickr dataset of more than 200k faces, the proposed formulation and joint ranking have made us achieve a hit rate of 0.420 at rank 100, significantly improving from 0.036 of the prior search scheme using attributes alone. We have also achieved an average running time of 0.0558 second by the proposed block-based indexing approach.
Making results fit into 40 characters: a study in document rewriting
Authors: Johannes Leveling, Gareth J. F. Jones
Venue: SIGIR
Year: 2012

With the increasing popularity of mobile and hand-held devices, automatic approaches for adapting results to the limited screen size of mobile devices are becoming more important. Traditional approaches for reducing the length of textual results include summarization and snippet extraction. In this study, we investigate document rewriting techniques which retain the meaning and readability of the original text. Evaluations on different document sets show that i) rewriting documents considerably reduces document length and thus, scrolling effort on devices with limited screen size, and ii) the rewritten documents have a higher readability.
A generalized hidden Markov model with discriminative training for query spelling correction
Authors: Yanen Li, Huizhong Duan, ChengXiang Zhai
Venue: SIGIR
Year: 2012

Query spelling correction is a crucial component of modern search engines. Existing methods in the literature for search query spelling correction have two major drawbacks. First, they are unable to handle certain important types of spelling errors, such as concatenation and splitting. Second, they cannot efficiently evaluate all the candidate corrections due to the complex form of their scoring functions, and a heuristic filtering step must be applied to select a working set of top-K most promising candidates for final scoring, leading to non-optimal predictions. In this paper we address both limitations and propose a novel generalized Hidden Markov Model with discriminative training that can not only handle all the major types of spelling errors, including splitting and concatenation errors, in a single unified framework, but also efficiently evaluate all the candidate corrections to ensure the finding of a globally optimal correction. Experiments on two query spelling correction datasets demonstrate that the proposed generalized HMM is effective for correcting multiple types of spelling errors. The results also show that it significantly outperforms the current approach for generating top-K candidate corrections, making it a better first-stage filter to enable any other complex spelling correction algorithm to have access to a better working set of candidate corrections as well as to cover splitting and concatenation errors, which no existing method in academic literature can correct.
Want a coffee?: predicting users' trails
Authors: Wen Li, Carsten Eickhoff, Arjen P. de Vries
Venue: SIGIR
Year: 2012

Twitter and Foursquare are two well-connected platforms for sharing information where growing numbers of users post location-related messages. In contrast to the longitude-latitude geotags commonly used online, e.g., on photos and tweets, new place-tags containing category information show more human-readable high-level information rather than a pair of coordinates. This grants an opportunity for better understanding users' physical locations which can be used as context to facilitate other applications, e.g., location context-aware advertisement. In this paper, we verify the assumption that users' current trails contain cues of their future routes. The results from the preliminary experiments show promising performance of a basic Markov Chain-based model.
Supporting efficient top-k queries in type-ahead search
Authors: Guoliang Li, Jiannan Wang, Chen Li, Jianhua Feng
Venue: SIGIR
Year: 2012

Type-ahead search can on-the-fly find answers as a user types in a keyword query. A main challenge in this search paradigm is the high-efficiency requirement that queries must be answered within milliseconds. In this paper we study how to answer top-k queries in this paradigm, i.e., as a user types in a query letter by letter, we want to efficiently find the k best answers. Instead of inventing completely new algorithms from scratch, we study challenges when adopting existing top-k algorithms in the literature that heavily rely on two basic list-access methods: random access and sorted access. We present two algorithms to support random access efficiently. We develop novel techniques to support efficient sorted access using list pruning and materialization. We extend our techniques to support fuzzy type-ahead search which allows minor errors between query keywords and answers. We report our experimental results on several real large data sets to show that the proposed techniques can answer top-k queries efficiently in type-ahead search.
Anticipatory search: using context to initiate search
Authors: Daniel J. Liebling, Paul N. Bennett, Ryen W. White
Venue: SIGIR
Year: 2012

Identifying content for which a user may search has a variety of applications, including ranking and recommendation. In this poster, we examine how pre-search context can be used to predict content that the user will seek before they have even specified a search query. We call this anticipatory search. Using a log-based approach, we compare different methods for predicting the content to be searched using different attributes of the pre-query context and behavioral signals from previous visitors to the most recent browse URL. Each method covers different cases and shows promise for query-free anticipatory search on the Web.
On the mathematical relationship between expected n-call@k and the relevance vs. diversity trade-off
Authors: Kar Wai Lim, Scott Sanner, Shengbo Guo
Venue: SIGIR
Year: 2012

It has been previously noted that optimization of the <i>n</i>-call@<i>k</i> relevance objective (i.e., a set-based objective that is 1 if at least <i>n</i> documents in a set of <i>k</i> are relevant, otherwise 0) encourages more result set diversification for smaller <i>n</i>, but this statement has never been formally quantified. In this work, we explicitly derive the mathematical relationship between <i>expected n-call@k</i> and the <i>relevance vs. diversity trade-off</i> --- through fortuitous cancellations in the resulting combinatorial optimization, we show the trade-off is a simple and intuitive function of <i>n</i> (notably independent of the result set size <i>k</i> e <i>n</i>), where diversification increases as <i>n</i> approaches 1.