OpenRaiser/Intern-Atlas huggingface dataset:
A comparison of time-aware ranking methods
Authors: Nattiya Kanhabua, Kjetil Nørvåg
Venue: SIGIR
Year: 2011

When searching a temporal document collection, e.g., news archives or blogs, the time dimension must be explicitly incorporated into a retrieval model in order to improve relevance ranking. Previous work has followed one of two main approaches: 1) a mixture model linearly combining textual similarity and temporal similarity, or 2) a probabilistic model generating a query from the textual and temporal part of a document independently. In this paper, we compare the effectiveness of different time-aware ranking methods by using a mixture model applied to all methods. Extensive evaluation is conducted using the New York Times Annotated Corpus, queries and relevance judgments obtained using the Amazon Mechanical Turk.
A large-scale study of the effect of training set characteristics over learning-to-rank algorithms
Authors: Evangelos Kanoulas, Stefan Savev, Pavel Metrikov, Virgiliu Pavlu, Javed A. Aslam
Venue: SIGIR
Year: 2011

In this work we describe the results of a large-scale study on the effect of the distribution of labels across the different grades of relevance in the training set on the performance of trained ranking functions. In a controlled experiment we generate a large number of training datasets wih different label distributions and employ three learning to rank algo- rithms over these datasets. We investigate the effect of these distributions on the accuracy of obtained ranking functions to give an insight into the manner training sets should be constructed.
A new look at old tricks: the fertile roots of current research
Authors: Paul B. Kantor
Venue: SIGIR
Year: 2011

As we face an explosion of potential new applications for the fundamental concepts and technologies of information retrieval, ranging from ad ranking to social media, from collaborative recommending to question answering systems, many researchers are spending unnecessary time reinventing ideas and relationships that are buried in the prehistory of information retrieval (which, for many researchers, means anything published before they entered graduate school). A lot of the ideas that surface as "new" in today's super-heated research environment have very firm roots in earlier developments in fields as diverse as citation analysis and pattern recognition. The purpose of this tutorial is to survey those roots, and their relation to the contemporary fruits on the tree of information retrieval, and to separate, as much as is possible in an era of increasing secrecy about methods, the problems to be solved, the algorithms for solving them, and the heuristics that are the bread and butter of a working operation. Participants will become familiar with roots in Pattern Analysis, Statistics, Information Science and other sources of key ideas that reappear in the current development of Information Retrieval as it applies to Search Engines, Social Media, and Collaborative Systems. They will be able to separate problems from algorithms, and algorithms from heuristics, in the application of these ideas to their own research and/or development activities. Course materials will be made available on a Web site two weeks prior to the tutorial. They will include links to relevant software; links to publications that will be discussed; and mechanisms for chat among the tutorial participants, before, during and after the tutorial.
Domain expert topic familiarity and search behavior
Authors: Sarvnaz Karimi, Falk Scholer, Adam Clark, Sadegh Kharazmi
Venue: SIGIR
Year: 2011

Users of information retrieval systems employ a variety of strategies when searching for information. One factor that can directly inﬂuence how searchers go about their information ﬁnding task is the level of familiaritywith a search topic. We investigate how the search behavior of domain experts changes based on their previous level of familiarity with a search topic, reporting on a user study of biomedical experts searching for a range of domain-speciﬁc material. The results of our study show that topic familiarity can inﬂuence the number of queries that are employed to complete a task, the types of queries that are entered, and the overall number of query terms. Our ﬁndings suggest that biomedical search systems should enable searching through a variety of querying modes, to support the diﬀerent search strategies that users were found to employ depending on their familiarity with the information that they are searching for.
Energy-price-driven query processing in multi-center web search engines
Authors: Enver Kayaaslan, Berkant Barla Cambazoglu, Roi Blanco, Flavio Paiva Junqueira, Cevdet Aykanat
Venue: SIGIR
Year: 2011

Concurrently processing thousands of web queries, each with a response time under a fraction of a second, necessitates maintaining and operating massive data centers. For large-scale web search engines, this translates into high energy consumption and a huge electric bill. This work takes the challenge to reduce the electric bill of commercial web search engines operating on data centers that are geographically far apart. Based on the observation that energy prices and query workloads show high spatio-temporal variation, we propose a technique that dynamically shifts the query workload of a search engine between its data centers to reduce the electric bill. Experiments on real-life query workloads obtained from a commercial search engine show that significant financial savings can be achieved by this technique.
Time-based relevance models
Authors: Mostafa Keikha, Shima Gerani, Fabio Crestani
Venue: SIGIR
Year: 2011

This paper addresses blog feed retrieval where the goal is to retrieve the most relevant blog feeds for a given user query. Since the retrieval unit is a blog, as a collection of posts, performing relevance feedback techniques and selecting the most appropriate documents for query expansion becomes challenging. By assuming time as an effective parameter on the blog posts content, we propose a time-based query expansion method. In this method, we select terms for expansion using most relevant days for the query, as opposed to most relevant documents. This provide us with more trustable terms for expansion. Our preliminary experiments on Blog08 collection shows that this method can outperform state of the art relevance feedback methods in blog retrieval.
BlogCast effect on information diffusion in a blogosphere
Authors: Sang-Wook Kim, Christos Faloutsos, Jiwoon Ha
Venue: SIGIR
Year: 2011

A blog service company provides a function named BlogCast that exposes quality posts on the blog main page to vitalize a blogosphere. This paper analyzes a new type of information diffusion via BlogCast. We show that there exists a strong halo effect in a blogosphere via thorough investigation on a huge volume of blog data.
Automatic boolean query suggestion for professional search
Authors: Youngho Kim, Jangwon Seo, W. Bruce Croft
Venue: SIGIR
Year: 2011

In professional search environments, such as patent search or legal search, search tasks have unique characteristics: 1) users interactively issue several queries for a topic, and 2) users are willing to examine many retrieval results, i.e., there is typically an emphasis on recall. Recent surveys have also verified that professional searchers continue to have a strong preference for Boolean queries because they provide a record of what documents were searched. To support this type of professional search, we propose a novel Boolean query suggestion technique. Specifically, we generate Boolean queries by exploiting decision trees learned from pseudo-labeled documents and rank the suggested queries using query quality predictors. We evaluate our algorithm in simulated patent and medical search environments. Compared with a recent effective query generation system, we demonstrate that our technique is effective and general.
Evaluating medical information retrieval
Authors: Bevan Koopman, Peter Bruza, Laurianne Sitbon, Michael Lawley
Venue: SIGIR
Year: 2011

This paper presents a framework for evaluating information retrieval of medical records. We use the BLULab corpus, a large collection of real-world de-identified medical records. The collection has been hand coded by clinical terminol- ogists using the ICD-9 medical classification system. The ICD codes are used to devise queries and relevance judge- ments for this collection. Results of initial test runs using a baseline IR system are provided. Queries and relevance judgements are online to aid further research in medical IR. Please visit: http://koopman.id.au/med_eval.
Cluster-based fusion of retrieved lists
Authors: Anna Khudyak Kozorovitzky, Oren Kurland
Venue: SIGIR
Year: 2011

Methods for fusing document lists that were retrieved in response to a query often use retrieval scores (or ranks) of documents in the lists. We present a novel probabilistic fusion approach that utilizes an additional source of rich information, namely, inter-document similarities. Specifically, our model integrates information induced from clusters of similar documents created across the lists with that produced by some fusion method that relies on retrieval scores (ranks). Empirical evaluation shows that our approach is highly effective for fusion. For example, the performance of our model is consistently better than that of the standard (effective) fusion method that it integrates. The performance also transcends that of standard fusion of re-ranked lists, where list re-ranking is based on clusters created from documents in the list.
Utilizing minimal relevance feedback for ad hoc retrieval
Authors: Eyal Krikon, Oren Kurland
Venue: SIGIR
Year: 2011

Using relevance feedback can significantly improve (ad hoc) retrieval effectiveness. Yet, if little feedback is available, effectively exploiting it is a challenge. To that end, we present a novel approach that utilizes document passages. Empirical evaluation demonstrates the merits of the approach.
Learning to rank from a noisy crowd
Authors: Abhimanu Kumar, Matthew Lease
Venue: SIGIR
Year: 2011

We study how to best use crowdsourced relevance judgments learning to rank [1, 7]. We integrate two lines of prior work: unreliable crowd-based binary annotation for binary classification [5, 3], and aggregating graded relevance judgments from reliable experts for ranking [7]. To model varying performance of the crowd, we simulate annotation noise with varying magnitude and distributional properties. Evaluation on three LETOR test collections reveals a striking trend contrary to prior studies: single labeling outperforms consensus methods in maximizing learner accuracy relative to annotator eýort. We also see surprising consistency of the learning curve across noise distributions, as well as greater challenge with the adversarial case for multi-class labeling.
ViewSer: enabling large-scale remote user studies of web search examination and interaction
Authors: Dmitry Lagun, Eugene Agichtein
Venue: SIGIR
Year: 2011

Web search behaviour studies, including eye-tracking studies of search result examination, have resulted in numerous insights to improve search result quality and presentation. Yet, eye tracking studies have been restricted in scale, due to the expense and the effort required. Furthermore, as the reach of the Web expands, it becomes increasingly important to understand how searchers around the world see and interact with the search results. To address both challenges, we introduce ViewSer, a novel methodology for performing web search examination studies remotely, at scale, and without requiring eye-tracking equipment. ViewSer operates by automatically modifying the appearance of a search engine result page, to clearly show one search result at a time as if through a viewport, while partially blurring the rest and allowing the participant to move the viewport naturally with a computer mouse or trackpad. Remarkably, the resulting result viewing and clickthrough patterns agree closely with unrestricted viewing of results, as measured by eye-tracking equipment, validated by a study with over 100 participants. We also explore applications of ViewSer to practical search tasks, such as analyzing the search result summary (snip- pet) attractiveness, result re-ranking, and evaluating snippet quality. These experiments could have only be done previously by tracking the eye movements for a small number of subjects in the lab. In contrast, our study was performed with over 100 participants, allowing us to reproduce and extend previous findings, establishing ViewSer as a valuable tool for large-scale search behavior experiments.
The effects of choice in routing relevance judgments
Authors: Edith Law, Paul N. Bennett, Eric Horvitz
Venue: SIGIR
Year: 2011

The emergence of human computation systems, including Mechanical Turk and games with a purpose, has made it feasible to distribute relevance judgment tasks to workers over the Web. Most human computation systems assign tasks to individuals randomly, and such assignments may match workers with tasks that they may be unqualified or unmotivated to perform. We compare two groups of workers, those given a choice of queries to judge versus those who are not, in terms of their self-rated competence and their actual performance. Results show that when given a choice of task, workers choose ones for which they have greater expertise, interests, confidence, and understanding.
Collaborative cyberporn filtering with collective intelligence
Authors: Lung-Hao Lee, Hsin-Hsi Chen
Venue: SIGIR
Year: 2011

This paper presents a user intent method to generate blacklists for collaborative cyberporn filtering. A novel porn detection framework that finds new pornographic web pages by mining user search behaviors is proposed. It employs users' clicks in search query logs to select the suspected web pages without extra human efforts to label data for training, and determines their categories with the help of URL host name and path information, but without web page content. We adopt an MSN porn data set to explore the effectiveness of our method. This user intent approach achieves high precision, while maintaining favorably low false positive rate. In addition, real-life filtering simulation reveals that our user intent method with its accumulative update strategy achieves 43.36% of blocking rate, while maintaining a steadily less than 7% of over-blocking rate.
Multi-layer graph-based semi-supervised learning for large-scale image datasets using mapreduce
Authors: Wen-Yu Lee, Liang-Chi Hsieh, Guan-Long Wu, Winston H. Hsu, Ya-Fan Su
Venue: SIGIR
Year: 2011

Semi-supervised learning is to exploit the vast amount of unlabeled data in the world. This paper proposes a scalable graph-based technique leveraging the distributed computing power of the MapReduce programming model. For a higher quality of learning, the paper also presents a multi-layer learning structure to unify both visual and textual information of image data during the learning process. Experimental results show the effectiveness of the proposed methods.
An investigation of decompounding for cross-language patent search
Authors: Johannes Leveling, Walid Magdy, Gareth J. F. Jones
Venue: SIGIR
Year: 2011

Decompounding has been found to improve information retrieval (IR) effectiveness in general domains for languages such as German or Dutch. We investigate if cross-language patent retrieval can profit from decompounding. This poses two challenges: i) There may be few resources such as parallel corpora available for training an machine translation system for a compounding language. ii) Patents have a specific writing style and vocabulary ("patentese"), which may affect the performance of decompounding and translation methods. Experiments on data from the CLEF-IP 2010 task show that decompounding patents for translation can overcome out-of-vocabulary problems (OOV) and that decompounding improves IR performance significantly for small training corpora.
Learning for graphs with annotated edges
Authors: Fan Li
Venue: SIGIR
Year: 2011

Automatic classification with graphs containing annotated edges is an interesting problem and has many potential applications. We present a risk minimization formulation that exploits the annotated edges for classification tasks. One major advantage of our approach compared to other methods is that the weight of each edge in the graph structures in our model, including both positive and negative weights, can be learned automatically from training data based on edge features. The empirical results show that our approach can lead to significantly improved classification performance compared to several baseline approaches.
Domain-specific information retrieval using rcommenders
Authors: Wei Li
Venue: SIGIR
Year: 2011

The continuing increase in the volume of information available in our daily lives is creating ever greater challenges for people to find personally useful information. One approach used to addressing this problem is Personalized Information Retrieval (PIR). PIR systems collect a user's personal information from both implicit and explicit sources to build a user profile with the objective of giving retrieval results which better meet their individual user information needs than a standard Information Retrieval (IR) system. However, in many situations there may be no opportunity to learn about the specific interests of a user and build a personal model when this user is querying on a new topic, e.g. when a user visits a museum or exhibition which is unrelated to their normal search interests. Under this condition, the experiences and behaviours of other previous users, who have made similar queries, could be used to build a model of user behavior in this domain. My PhD proposes to focus on the development of new and innovative methods of domain-specific IR. My work seeks to combine recommender algorithms trained using previous search behaviours from different searchers with a standard ranked IR method to form a domain-specific IR model to improve the search effectiveness for a user entering a query without personal prior search history on this topic. The challenges for my work are: how to provide users better results; how to train and evaluate the methods proposed in my work.
Semantic tag recommendation using concept model
Authors: Chenliang Li, Anwitaman Datta, Aixin Sun
Venue: SIGIR
Year: 2011

The common tags given by multiple users to a particular document are often semantically relevant to the document and each tag represents a specific topic. In this paper, we attempt to emulate human tagging behavior to recommend tags by considering the concepts contained in documents. Specifically, we represent each document using a few most relevant concepts contained in the document, where the concept space is derived from Wikipedia. Tags are then recommended based on the tag concept model derived from the annotated documents of each tag. Evaluated on a Delicious dataset of more than 53K documents, the proposed technique achieved comparable tag recommendation accuracy as the state-of-the-art, while yielding an order of magnitude speed-up.
MSSF: a multi-document summarization framework based on submodularity
Authors: Jingxuan Li, Lei Li, Tao Li
Venue: SIGIR
Year: 2011

Multi-document summarization aims to distill the most representative information from a set of documents to generate a summary. Given a set of documents as input, most of existing multi-document summarization approaches utilize different sentence selection techniques to extract a set of sentences from the document set as the summary. The submodularity hidden in textual-unit similarity motivates us to incorporate this property into our solution to multi-document summarization tasks. In this poster, we propose a new principled and versatile framework for different multi-document summarization tasks using the submodular function [8].
Exploiting endorsement information and social influence for item recommendation
Authors: Cheng-Te Li, Shou-De Lin, Man-Kwan Shan
Venue: SIGIR
Year: 2011

Social networking services possess two features: (1) capturing the social relationships among people, represented by the social network, and (2) allowing users to express their preferences on different kinds of items (e.g. photo, celebrity, pages) through endorsing buttons, represented by a kind of endorsement bipartite graph. In this work, using such information, we propose a novel recommendation method, which leverages the viral marketing in the social network and the wisdom of crowds from endorsement network. Our recommendation consists of two parts. First, given some query terms describing user's preference, we find a set of targeted influencers who have the maximum activation probability on those nodes related to the query terms in the social network. Second, based on the derived targeted influencers as key experts, we recommend items via the endorsement network. We conduct the experiments on DBLP co-authorship social network with author-reference data as the endorsement network. The results show our method can achieve effective recommendations.
Optimizing multimodal reranking for web image search
Authors: Hao Li, Meng Wang, Zhisheng Li, Zheng-Jun Zha, Jialie Shen
Venue: SIGIR
Year: 2011

In this poster, we introduce a web image search reranking approach with exploring multiple modalities. Diff erent from the conventional methods that build graph with one feature set for reranking, our approach integrates multiple feature sets that describe visual content from different aspects. We simultaneously integrate the learning of relevance scores, the weighting of different feature sets, the distance metric and the scaling for each feature set into a unified scheme. Experimental results on a large data set that contains more than 1,100 queries and 1 million images demonstrate the effectiveness of our approach.
Product comparison using comparative relations
Authors: Si Li, Zheng-Jun Zha, Zhaoyan Ming, Meng Wang, Tat-Seng Chua, Jun Guo, Weiran Xu
Venue: SIGIR
Year: 2011

This paper proposes a novel Product Comparison approach. The comparative relations between products are first mined from both user reviews on multiple review websites and community-based question answering pairs containing product comparison information. A unified graph model is then developed to integrate the resultant comparative relations for product comparison. Experiments on popular electronic products show that the proposed approach outperforms the state-of-the-art methods.
Disambiguating biomedical acronyms using EMIM
Authors: Nut Limsopatham, Rodrygo L. T. Santos, Craig Macdonald, Iadh Ounis
Venue: SIGIR
Year: 2011

Expanding a query with acronyms or their corresponding 'long-forms' has not been shown to provide consistent improvements in the biomedical IR literature. The major open issue with expanding acronyms in a query is their inherent ambiguity, as an acronym can refer to multiple long-forms. At the same time, a long-form identified in a query can be expanded with its acronym(s); however, some of these may be also ambiguous and lead to poor retrieval performance. In this work, we propose the use of the EMIM (Expected Mutual Information Measure) between a long-form and its abbreviated acronym to measure ambiguity. We experiment with expanding both acronyms and long-forms identified in the queries from the adhoc task of the TREC 2004 Genomics track. Our preliminary analysis shows the potential of both acronym and long-form expansions for biomedical IR.