OpenRaiser/Intern-Atlas huggingface dataset:
Intent-aware search result diversification
Authors: Rodrygo L. T. Santos, Craig Macdonald, Iadh Ounis
Venue: SIGIR
Year: 2011

Search result diversification has gained momentum as a way to tackle ambiguous queries. An effective approach to this problem is to explicitly model the possible aspects underlying a query, in order to maximise the estimated relevance of the retrieved documents with respect to the different aspects. However, such aspects themselves may represent information needs with rather distinct intents (e.g., informational or navigational). Hence, a diverse ranking could benefit from applying intent-aware retrieval models when estimating the relevance of documents to different aspects. In this paper, we propose to diversify the results retrieved for a given query, by learning the appropriateness of different retrieval models for each of the aspects underlying this query. Thorough experiments within the evaluation framework provided by the diversity task of the TREC 2009 and 2010 Web tracks show that the proposed approach can significantly improve state-of-the-art diversification approaches.
On the suitability of diversity metrics for learning-to-rank for diversity
Authors: Rodrygo L. T. Santos, Craig Macdonald, Iadh Ounis
Venue: SIGIR
Year: 2011

An optimally diverse ranking should achieve the maximum coverage of the aspects underlying an ambiguous or under-specified query, with minimum redundancy with respect to the covered aspects. Although evaluation metrics that re-ward coverage and penalise redundancy provide intuitive ob-jective functions for learning a diverse ranking, it is unclear whether they are the most effective. In this paper, we con-trast the suitability of relevance and diversity metrics as ob-jective functions for learning a diverse ranking. Our results in the context of the diversity task of the TREC 2009 and 2010 Web tracks show that diversity metrics are not neces-sarily better suited for guiding a learning approach. More-over, the suitability of these metrics is compromised as they try to penalise redundancy during the learning process.
How diverse are web search results?
Authors: Rodrygo L. T. Santos, Craig Macdonald, Iadh Ounis
Venue: SIGIR
Year: 2011

Search result diversification has recently gained attention as a means to tackle ambiguous queries. While query ambiguity is of particular concern for the short queries commonly observed in a Web search scenario, it is unclear how much diversity is actually promoted by Web search engines (WSEs). In this paper, we assess the diversification performance of two leading WSEs in the context of the diversity task of the TREC 2009 and 2010 Web tracks. Our results show that these WSEs perform effectively for queries with multiple interpretations, but not for those open to multiple aspects related to a single interpretation. Moreover, by deploying a state-of-the-art diversification approach based on query suggestions from these WSEs themselves, we show that their diversification performance can be further improved.
Evaluating the synergic effect of collaboration in information seeking
Authors: Chirag Shah, Roberto I. González-Ibáñez
Venue: SIGIR
Year: 2011

It is typically expected that when people work together, they can often accomplish goals that are difficult or even impossible for individuals. We consider this notion of the group achieving more than the sum of all individuals' achievements to be the synergic effect in collaboration. Similar expectation exists for people working in collaboration for information seeking tasks. We, however, lack a methodology and appropriate evaluation metrics for studying and measuring the synergic effect. In this paper we demonstrate how to evaluate this effect and discuss what it means to various collaborative information seeking (CIS) situations. We present a user study with four different conditions: single user, pair of users at the same computer, pair of users at different computers and co-located, and pair of users remotely located. Each of these individuals or pairs was given the same task of information seeking and usage for the same amount of time. We then combined the outputs of single independent users to form artificial pairs, and compared against the real pairs. Not surprisingly, participants using different computers (co-located or remotely located) were able to cover more information sources than those using a single computer (single user or a pair). But more interestingly, we found that real pairs with their own computers (co-located or remotely located) were able to cover more unique and useful information than that of the artificially created pairs. This indicates that those working in collaboration achieved something greater and better than what could be achieved by adding independent users, thus, demonstrating the synergic effect. Remotely located real teams were also able to formulate a wider range of queries than those pairs that were co-located or artificially created. This shows that the collaborators working remotely were able to achieve synergy while still being able to think and work independently. Through the experiments and measurements presented here, we have also contributed a unique methodology and an evaluation metric for CIS.
A large scale machine learning system for recommending heterogeneous content in social networks
Authors: Yanxin Shi, David Ye, Andrey Goder, Srinivas Narayanan
Venue: SIGIR
Year: 2011

The goal of the Facebook recommendation engine is to compare and rank heterogeneous types of content in order to find the most relevant recommendations based on user preference and page context. The challenges for such a recommendation engine include several aspects: 1) the online queries being processed are at very large scale; 2) with new content types and new user-generated content constantly added to the system, the candidate object set and underlying data distribution change rapidly; 3) different types of content usually have very distinct characteristics, which makes generic feature engineering difficult; and 4) unlike a search engine that can capture intention of users based on their search queries, our recommendation engine needs to focus more on users' profile and interests, past behaviors and current actions in order to infer their cognitive states. In this presentation, we would like to introduce an effective, scalable, online machine learning framework we developed in order to address the aforementioned challenges. We also want to discuss the insights, approaches and experiences we have accumulated during our research and development process.
What-you-retrieve-is-what-you-see: a preliminary cyber-physical search engine
Authors: Lidan Shou, Ke Chen, Gang Chen, Chao Zhang, Yi Ma, Xian Zhang
Venue: SIGIR
Year: 2011

The cyber-physical systems (CPS) are envisioned as a class of real-time systems integrating the computing, communication and storage facilities with monitoring and control of the physical world. One interesting CPS application in the mobile Internet is to provide Web search "on the spot" regarding the physical world that a user sees, or literally WYRIWYS (What-You-Retrieve-Is-What-You-See). The objective of our work is to develop server/browser software for supporting WYRIWYS search in our prototype cyber-physical search engine. A WYRIWYS search retrieves visible Web objects and ranks them by their cyber-physical relevances (term, visual, spatial, temporal etc.). This work is distinguished from previous LWS as it provides quality Web search geared with the physical world. Therefore it suggests a very promising solution to cyber-physical Web search.
Machine learning for information retrieval
Authors: Luo Si, Rong Jin
Venue: SIGIR
Year: 2011

In recent years, we have witnessed successful application of machine learning techniques to a wide range of information retrieval problems, including Web search engines, recommendation systems, online advertising, etc. It is thus critical for researchers in the information retrieval community to understand the core machine learning techniques. In order to accommodate audiences with different levels of understanding of machine learning, we divide this tutorial into two sessions: the first session will focus on basic machine learning concepts and tools; in the second session, we will introduce more advanced topics in machine learning, and will present recent developments in machine learning and its application to information retrieval. Each season is self-contained.
User behavior in zero-recall ecommerce queries
Authors: Gyanit Singh, Nish Parikh, Neel Sundaresan
Venue: SIGIR
Year: 2011

User expectation and experience for web search and eCommerce (product) search are quite different. Product descriptions are concise as compared to typical web documents. User expectation is more specific to find the right product. The difference in the publisher and searcher vocabulary (in case of product search the seller and the buyer vocabulary) combined with the fact that there are fewer products to search over than web documents result in observable numbers of searches that return no results (zero recall searches). In this paper we describe a study of zero recall searches. Our study is focused on eCommerce search and uses data from a leading eCommerce site's user click stream logs. There are 3 main contributions of our study: 1) The cause of zero recall searches; 2) A study of user's reaction and recovery from zero recall; 3) A study of differences in behavior of power users versus novice users to zero recall searches.
Ranking tags in resource collections
Authors: Dimitrios Skoutas, Mohammad Alrifai
Venue: SIGIR
Year: 2011

We examine different tag ranking strategies for constructing tag clouds to represent collections of tagged objects. The proposed methods are based on random walk on graphs, diversification, and rank aggregation, and they are empirically evaluated on a data set of tagged images from Flickr.
A model for expert finding in social networks
Authors: Elena Smirnova
Venue: SIGIR
Year: 2011

Expert finding is a task of finding knowledgeable people on a given topic. State-of-the-art expertise retrieval algorithms identify matching experts based on analysis of textual content of documents experts are associated with. While powerful, these models ignore social structure that might be available. In this paper, we develop a Bayesian hierarchical model for expert finding that accounts for both social relationships and content. The model assumes that social links are determined by expertise similarity between candidates. We demonstrate the improved retrieval performance of our model over the baseline on a realistic data set.
Post-ranking query suggestion by diversifying search results
Authors: Yang Song, Dengyong Zhou, Li-wei He
Venue: SIGIR
Year: 2011

Query suggestion refers to the process of suggesting related queries to search engine users. Most existing researches have focused on improving the relevance of suggested queries. In this paper, we introduce the concept of diversifying the content of the search results from suggested queries while keeping the suggestion relevant. Our framework first retrieves a set of query candidates from search engine logs using random walk and other techniques. We then re-rank the suggested queries by ranking them in the order which maximizes the diversification function that measures the difference between the original search results and the results from suggested queries. The diversification function we proposed includes features like ODP category, URL and domain similarity and so on. One important outcome from our research which contradicts with most existing researches is that, with the increase of suggestion relevance, the similarity between the queries actually decreases. Experiments are conducted on a large set of human-labeled data, which is randomly sampled from a commercial search engine's log. Results indicate that the post-ranking framework significantly improves the relevance of suggested queries by comparing to existing models.
Query term ranking based on search results overlap
Authors: Wei Song, Yu Zhang, Yubin Xie, Ting Liu, Sheng Li
Venue: SIGIR
Year: 2011

In this paper, we propose a method to rank and assign weights to query terms according to their impact on the topic of the query. We use Search Result Overlap Ratio (SROR) to quantify the overlap of the search results of the full query and a shorten query after removing one term. Intuitively, if the overlap is small, it indicates a big topic shift and the removed term should be discriminative and important. The SROR could be used for measuring query term importance with a search engine automatically. By this way, learning based models could be trained based on a large number of automatically labeled instances and make predictions for future queries efficiently.
An event-centric model for multilingual document similarity
Authors: Jannik Strötgen, Michael Gertz, Conny Junghans
Venue: SIGIR
Year: 2011

Document similarity measures play an important role in many document retrieval and exploration tasks. Over the past decades, several models and techniques have been developed to determine a ranked list of documents similar to a given query document. Interestingly, the proposed approaches typically rely on extensions to the vector space model and are rarely suited for multilingual corpora.
Picasso - to sing, you must close your eyes and draw
Authors: Aleksandar Stupar, Sebastian Michel
Venue: SIGIR
Year: 2011

We study the problem of automatically assigning appropriate music pieces to a picture or, in general, series of pictures. This task, commonly referred to as soundtrack suggestion, is non-trivial as it requires a lot of human attention and a good deal of experience, with master pieces distinguished, e.g., with the Academy Award for Best Original Score. We put forward PICASSO to solve this task in a fully automated way. PICASSO makes use of genuine samples obtained from first-class contemporary movies. Hence, the training set can be arbitrarily large and is also inexpensive to obtain but still provides an excellent source of information. At query time, PICASSO employs a three-level algorithm. First, it selects for a given query image a ranking of the most similar screenshots taken, and subsequently, selects for each screenshot the most similar songs to the music played in the movie when the screenshot was taken. Last, it issues a top-K aggregation algorithm to find the overall best suitable songs available. We have created a large training set consisting of over 40,000 image/soundtrack samples obtained from 28 movies and evaluated the suitability of PICASSO by means of a user study.
Visualizing and querying semantic social networks
Authors: Aixin Sun, Anwitaman Datta, Ee-Peng Lim, Kuiyu Chang
Venue: SIGIR
Year: 2011

We demonstrate SSNetViz that is developed for integrating, visualizing and querying heterogeneous semantic social networks obtained from multiple information sources. A semantic social network refers to a social network graph with multi-typed nodes and links. We demonstrate various innovative features of SSNetViz with social networks from three information sources covering a similar set of entities and relationships in terrorism domain.
Towards effective short text deep classification
Authors: Xinruo Sun, Haofen Wang, Yong Yu
Venue: SIGIR
Year: 2011

Recently, more and more short texts (e.g., ads, tweets) appear on the Web. Classifying short texts into a large taxonomy like ODP or Wikipedia category system has become an important mining task to improve the performance of many applications such as contextual advertising and topic detection for micro-blogging. In this paper, we propose a novel multi-stage classification approach to solve the problem. First, explicit semantic analysis is used to add more features for both short texts and categories. Second, we leverage information retrieval technologies to fetch the most relevant categories for an input short text from thousands of candidates. Finally, a SVM classifier is applied on only a few selected categories to return the final answer. Our experimental results show that the proposed method achieved significant improvements on classification accuracy compared with several existing state of art approaches.
A bipartite graph based social network splicing method for person name disambiguation
Authors: Jintao Tang, Qin Lu, Ting Wang, Ji Wang, Wenjie Li
Venue: SIGIR
Year: 2011

The key issue of person name disambiguation is to discover different namesakes in massive web documents rather than simply cluster documents by using textual features. In this paper, we describe a novel person name disambiguation method based on social networks to effectively identify namesakes. The social network snippets in each document are extracted. Then, the namesakes are identified via splicing the social networks of each namesake by using the snippets as a bipartite graph. Experimental results show that our method achieves better result than the top performance of WePS-2 in identifying different namesakes.
Recommending interesting activity-related local entities
Authors: Jie Tang, Ryen W. White, Peter Bailey
Venue: SIGIR
Year: 2011

When searching for entities with a strong local character (e.g., a museum), people may also be interested in discovering proximal activity-related entities (e.g., a café). Geographical proximity is a necessary, but not sufficient, qualifier for recommending other entities such that they are related in a useful manner (e.g., interest in a fish market does not imply interest in nearby bookshops, but interest in other produce stores is more likely). We describe and evaluate methods to identify such activity-related local entities.
Posting list intersection on multicore architectures
Authors: Shirish Tatikonda, Berkant Barla Cambazoglu, Flavio Paiva Junqueira
Venue: SIGIR
Year: 2011

In current commercial Web search engines, queries are processed in the conjunctive mode, which requires the search engine to compute the intersection of a number of posting lists to determine the documents matching all query terms. In practice, the intersection operation takes a significant fraction of the query processing time, for some queries dominating the total query latency. Hence, efficient posting list intersection is critical for achieving short query latencies. In this work, we focus on improving the performance of posting list intersection by leveraging the compute capabilities of recent multicore systems. To this end, we consider various coarse-grained and fine-grained parallelization models for list intersection. Specifically, we present an algorithm that partitions the work associated with a given query into a number of small and independent tasks that are subsequently processed in parallel. Through a detailed empirical analysis of these alternative models, we demonstrate that exploiting parallelism at the finest-level of granularity is critical to achieve the best performance on multicore systems. On an eight-core system, the fine-grained parallelization method is able to achieve more than five times reduction in average query processing time while still exploiting the parallelism for high query throughput.
What deliberately degrading search quality tells us about discount functions
Authors: Paul Thomas, Timothy Jones, David Hawking
Venue: SIGIR
Year: 2011

Deliberate degradation of search results is a common tool in user experiments. We degrade high-quality search results by inserting non-relevant documents at different ranks. The effect of these manipulations, on a number of commonly-used metrics, is counter-intuitive: the discount functions implicit in P@k, MRR, NDCG, and others do not account for the true relationship between rank and value to the user. We propose an alternative, based on visibility data. Categories and Subject Descriptors: H.3.4 [Information
Active learning to maximize accuracy vs. effort in interactive information retrieval
Authors: Aibo Tian, Matthew Lease
Venue: SIGIR
Year: 2011

We consider an interactive information retrieval task in which the user is interested in finding several to many relevant documents with minimal effort. Given an initial document ranking, user interaction with the system produces relevance feedback (RF) which the system then uses to revise the ranking. This interactive process repeats until the user terminates the search. To maximize accuracy relative to user effort, we propose an active learning strategy. At each iteration, the document whose relevance is maximally uncertain to the system is slotted high into the ranking in order to obtain user feedback for it. Simulated feedback on the Robust04 TREC collection shows our active learning approach dominates several standard RF baselines relative to the amount of feedback provided by the user. Evaluation on Robust04 under noisy feedback and on LETOR collections further demonstrate the effectiveness of active learning, as well as value of negative feedback in this task scenario.
Effect of different docid orderings on dynamic pruning retrieval strategies
Authors: Nicola Tonellotto, Craig Macdonald, Iadh Ounis
Venue: SIGIR
Year: 2011

Document-at-a-time (DAAT) dynamic pruning strategies for information retrieval systems such as MaxScore and Wand can increase querying efficiency without decreasing effectiveness. Both work on posting lists sorted by ascending document identifier (docid). The order in which docids are assigned -- and hence the order of postings in the posting lists -- is known to have a noticeable impact on posting list compression. However, the resulting impact on dynamic pruning strategies is not well understood. In this poster, we examine the impact on the efficiency of these strategies across different docid orderings, by experimenting using the TREC ClueWeb09 corpus. We find that while the number of postings scored by dynamic pruning strategies do not markedly vary for different docid orderings, the ordering still has a marked impact on mean query response time. Moreover, when docids are assigned by lexicographical URL ordering, the benefit to response time for is more pronounced for Wand than for MaxScore.
Ad hoc IR: not much room for improvement
Authors: Andrew Trotman, David Keeler
Venue: SIGIR
Year: 2011

Ranking function performance reached a plateau in 1994. The reason for this is investigated. First the performance of BM25 is measured as the proportion of queries satisfied on the first page of 10 results -- it performs well. The performance is then compared to human performance. They perform comparably. The conclusion is there isn't much room for ranking function improvement.
Hypergeometric language models for republished article finding
Authors: Manos Tsagkias, Maarten de Rijke, Wouter Weerkamp
Venue: SIGIR
Year: 2011

Republished article finding is the task of identifying instances of articles that have been published in one source and republished more or less verbatim in another source, which is often a social media source. We address this task as an ad hoc retrieval problem, using the source article as a query. Our approach is based on language modeling. We revisit the assumptions underlying the unigram language model taking into account the fact that in our setup queries are as long as complete news articles. We argue that in this case, the underlying generative assumption of sampling words from a document with replacement, i.e., the multinomial modeling of documents, produces less accurate query likelihood estimates.