OpenRaiser/Intern-Atlas huggingface dataset:
Towards a Game-Theoretic Framework for Information Retrieval
Authors: ChengXiang Zhai
Venue: SIGIR
Year: 2015

The task of information retrieval (IR) has traditionally been defined as to rank a collection of documents in response to a query. While this definition has enabled most research progress in IR so far, it does not model accurately the actual retrieval task in a real IR application, where users tend to be engaged in an interactive process with multipe queries, and optimizing the overall performance of an IR system on an entire search session is far more important than its performance on an individual query. In this talk, I will present a new game-theoretic formulation of the IR problem where the key idea is to model information retrieval as a process of a search engine and a user playing a cooperative game, with a shared goal of satisfying the user's information need (or more generally helping the user complete a task) while minimizing the user's effort and the resource overhead on the retrieval system. Such a game-theoretic framework offers several benefits. First, it naturally suggests optimization of the overall utility of an interactive retrieval system over a whole search session, thus breaking the limitation of the traditional formulation that optimizes ranking of documents for a single query. Second, it models the interactions between users and a search engine, and thus can optimize the collaboration of a search engine and its users, maximizing the "combined intelligence" of a system and users. Finally, it can serve as a unified framework for optimizing both interactive information retrieval and active relevance judgment acquisition through crowdsourcing. I will discuss how the new framework can not only cover several emerging directions in current IR research as special cases, but also open up many interesting new research directions in IR.
When Personalization Meets Conformity: Collective Similarity based Multi-Domain Recommendation
Authors: Xi Zhang, Jian Cheng, Shuang Qiu, Zhenfeng Zhu, Hanqing Lu
Venue: SIGIR
Year: 2015

Existing recommender systems place emphasis on personalization to achieve promising accuracy. However, in the context of multiple domain, users are likely to seek the same behaviors as domain authorities. This conformity effect provides a wealth of prior knowledge when it comes to multi-domain recommendation, but has not been fully exploited. In particular, users whose behaviors are significant similar with the public tastes can be viewed as domain authorities. To detect these users meanwhile embed conformity into recommendation, a domain-specific similarity matrix is intuitively employed. Therefore, a collective similarity is obtained to leverage the conformity with personalization. In this paper, we establish a Collective Structure Sparse Representation(CSSR) method for multi-domain recommendation. Based on adaptive $k$-Nearest-Neighbor framework, we impose the lasso and group lasso penalties as well as least square loss to jointly optimize the collective similarity. Experimental results on real-world data confirm the effectiveness of the proposed method.
Compact Snippet Caching for Flash-based Search Engines
Authors: Rui Zhang, Pengyu Sun, Jiancong Tong, Rebecca Jane Stones, Gang Wang, Xiaoguang Liu
Venue: SIGIR
Year: 2015

In response to a user query, search engines return the top-k relevant results, each of which contains a small piece of text, called a snippet, extracted from the corresponding document. Obtaining a snippet is time consuming as it requires both document retrieval (disk access) and string matching (CPU computation), so caching of snippets is used to reduce latency. With the trend of using flash-based solid state drives (SSDs) instead of hard disk drives for search engine storage, the bottleneck of snippet generation shifts from I/O to computation. We propose a simple, but effective method for exploiting this trend, which we call fragment caching: instead of caching the whole snippet, we only cache snippet metadata which describe how to retrieve the snippet from the document. While this approach increases I/O time, the cost is insignificant on SSDs. The major benefit of fragment caching is the ability to cache the same snippets (without loss of quality) while only using a fraction of the memory the traditional method requires. In our experiments, we find around 10 times less memory is required to achieve comparable snippet generation times for dynamic memory, and we consistently achieve a vastly greater hit ratio for static caching.
Sub-document Timestamping of Web Documents
Authors: Yue Zhao, Claudia Hauff
Venue: SIGIR
Year: 2015

Knowledge about a (Web) document's creation time has been shown to be an important factor in various temporal information retrieval settings. Commonly, it is assumed that such documents were created at a single point in time. While this assumption may hold for news articles and similar document types, it is a clear oversimplification for general Web documents. In this paper, we investigate to what extent (i) this simplifying assumption is violated for a corpus of Web documents, and, (ii) it is possible to accurately estimate the creation time of individual Web documents' components (so-called sub-documents).
Learning to Reweight Terms with Distributed Representations
Authors: Guoqing Zheng, Jamie Callan
Venue: SIGIR
Year: 2015

Term weighting is a fundamental problem in IR research and numerous weighting models have been proposed. Proper term weighting can greatly improve retrieval accuracies, which essentially involves two types of query understanding: interpreting the query and judging the relative contribution of the terms to the query. These two steps are often dealt with separately, and complicated yet not so effective weighting strategies are proposed. In this paper, we propose to address query interpretation and term weighting in a unified framework built upon distributed representations of words from recent advances in neural network language modeling. Specifically, we represent term and query as vectors in the same latent space, construct features for terms using their word vectors and learn a model to map the features onto the defined target term weights. The proposed method is simple yet effective. Experiments using four collections and two retrieval models demonstrates significantly higher retrieval accuracies than baseline models.
DUMPLING: A Novel Dynamic Search Engine
Authors: Andrew Jie Zhou, Jiyun Luo, Hui Yang
Venue: SIGIR
Year: 2015

In this demo paper, we introduce a new search engine that supports Information Retrieval (IR) in a dynamic setting. A dynamic search engine distinguishes itself by handling rich interactions and temporal dependency among the queries in a session or for a task. The proposed search engine is called Dumpling, named after the development team's favorite food. It implements state-of-the-art dynamic search algorithms and provides: (i) a dynamic search toolkit by integrating the Query Change Retrieval Model (QCM) and the Win-win search algorithm; (ii) a user-friendly interface supporting side-by-side comparison of search results given by a state-of-the-art static search algorithm and the proposed dynamic search algorithms; (iii) and APIs for developers to apply the dynamic search algorithms to index and search over custom datasets. Dumpling is developed under the umbrella of a bigger project in the DARPA Memex program to crawl and search the dark web to support law enforcement and national security.
From Design to Analysis: Conducting Controlled Laboratory Experiments with Users
Authors: Diane Kelly, Anita Crescenzi
Venue: SIGIR
Year: 2016

This full-day tutorial provides general instruction about the design of controlled laboratory experiments that are conducted in order to better understand human information interaction and retrieval. Different data collection methods and procedures are described, with an emphasis on self-report measures and scales. This tutorial also introduces the use of statistical power analysis for sample size estimation and introduces and demonstrate two data analysis procedures, Multilevel Modeling and Structural Equation Modeling, that allow for examination of the whole set of variables present in interactive information retrieval (IIR) experiments, along with their various effect sizes. The goals of the tutorial are (1) to increase participants? understanding of the uses of controlled laboratory experiments with human participants; (2) to increase participants? understanding of the technical vocabulary and procedures associated with such experiments and (2) to increase participants? confidence in conducting and evaluating IIR experiments. Ultimately, we hope our tutorial will increase research capacity and research quality in IR by providing instruction about best practices to those contemplating interactive IR experiments.
Quit While Ahead: Evaluating Truncated Rankings
Authors: Fei Liu, Alistair Moffat, Timothy Baldwin, Xiuzhen Zhang
Venue: SIGIR
Year: 2016

Many types of search tasks are answered through the computation of a ranked list of suggested answers. We re-examine the usual assumption that answer lists should be as long as possible, and suggest that when the number of matching items is potentially small -- perhaps even zero -- it may be more helpful to "quit while ahead", that is, to truncate the answer ranking earlier rather than later. To capture this effect, metrics are required which are attuned to the length of the ranking, and can handle cases in which there are no relevant documents. In this work we explore a generalized approach for representing truncated result sets, and propose modifications to a number of popular evaluation metrics.
Topic Model based Privacy Protection in Personalized Web Search
Authors: Wasi Uddin Ahmad, Md. Masudur Rahman, Hongning Wang
Venue: SIGIR
Year: 2016

Modern search engines utilize users' search history for personalization, which provides more effective, useful and relevant search results. However, it also has the potential risk of revealing users' privacy by identifying their underlying intention from their logged search behaviors. To address this privacy issue, we proposed a Topic-based Privacy Protection solution on client side. In our solution, each user query will be submitted with k additional cover queries, which will act as a proxy to disguise users' intent from a search engine. The set of cover queries are generated in a controlled way so that each query carries similar uncertainty to randomize a user's search history while still providing necessary utility for the search engine to perform personalization. We used statistical topic models to infer topics from the original user query and generated cover queries of similar entropy but from unrelated topics. Extensive experiments are performed on AOL search log and the promising results demonstrated the effectiveness of our solution.
Improving Language Estimation with the Paragraph Vector Model for Ad-hoc Retrieval
Authors: Qingyao Ai, Liu Yang, Jiafeng Guo, W. Bruce Croft
Venue: SIGIR
Year: 2016

Incorporating topic level estimation into language models has been shown to be beneficial for information retrieval (IR) models such as cluster-based retrieval and LDA-based document representation. Neural embedding models, such as paragraph vector (PV) models, on the other hand have shown their effectiveness and efficiency in learning semantic representations of documents and words in multiple Natural Language Processing (NLP) tasks. However, their effectiveness in information retrieval is mostly unknown. In this paper, we study how to effectively use the PV model to improve ad-hoc retrieval. We propose three major improvements over the original PV model to adapt it for the IR scenario: (1) we use a document frequency-based rather than the corpus frequency-based negative sampling strategy so that the importance of frequent words will not be suppressed excessively; (2) we introduce regularization over the document representation to prevent the model overfitting short documents along with the learning iterations; and (3) we employ a joint learning objective which considers both the document-word and word-context associations to produce better word probability estimation. By incorporating this enhanced PV model into the language modeling framework, we show that it can significantly outperform the state-of-the-art topic enhanced language models.
Network-Aware Recommendations of Novel Tweets
Authors: Noor Aldeen Alawad, Aris Anagnostopoulos, Stefano Leonardi, Ida Mele, Fabrizio Silvestri
Venue: SIGIR
Year: 2016

With the rapid proliferation of microblogging services such as Twitter, a large number of tweets is published everyday often making users feel overwhelmed with information. Helping these users to discover potentially interesting tweets is an important task for such services. In this paper, we present a novel tweet-recommendation approach, which exploits network, content, and retweet analyses for making recommendations of tweets. The idea is to recommend tweets that are not visible to the user (i.e., they do not appear in the user timeline) because nobody in her social circles published or retweeted them. To do that, we create the user's ego-network up to depth two and apply the transitivity property of the friends-of-friends relationship to determine interesting recommendations, which are then ranked to best match the user's interests. Experimental results demonstrate that our approach improves the state-of-the-art technique.
EveTAR: A New Test Collection for Event Detection in Arabic Tweets
Authors: Hind A. Al-Merekhi, Maram Hasanain, Tamer Elsayed
Venue: SIGIR
Year: 2016

Research on event detection in Twitter is often obstructed by the lack of publicly-available evaluation mechanisms such as test collections; this problem is more severe when considering the scarcity of them in languages other than English. In this paper, we present EveTAR, the first publicly-available test collection for event detection in Arabic tweets. The collection includes a crawl of 590M Arabic tweets posted in a month period and covers 66 significant events (in 8 different categories) for which more than 134k relevance judgments were gathered using crowdsourcing with high average inter-annotator agreement (Kappa value of 0.6). We demonstrate the usability of the collection by evaluating 3 state-of-the-art event detection algorithms. The collection is also designed to support other retrieval tasks, as we show in our experiments with ad-hoc search systems.
The Data Stack in Information Retrieval
Authors: Omar Alonso
Venue: SIGIR
Year: 2016

I propose to look at information retrieval applications from the perspective of the data stack infrastructure that is needed in research prototypes and production systems.
A Visual Analytics Approach for What-If Analysis of Information Retrieval Systems
Authors: Marco Angelini, Nicola Ferro, Giuseppe Santucci, Gianmaria Silvello
Venue: SIGIR
Year: 2016

We present the innovative visual analytics approach of the VATE system, which eases and makes more effective the experimental evaluation process by introducing the what-if analysis. The what-if analysis is aimed at estimating the possible effects of a modification to an IR system to select the most promising fixes before implementing them, thus saving a considerable amount of effort. VATE builds on an analytical framework which models the behavior of the systems in order to make estimations, and integrates this analytical framework into a visual part which, via proper interaction and animations, receives input and provides feedback to the user.
Predicting User Engagement with Direct Displays Using Mouse Cursor Information
Authors: Ioannis Arapakis, Luis A. Leiva
Venue: SIGIR
Year: 2016

Predicting user engagement with direct displays (DD) is of paramount importance to commercial search engines, as well as to search performance evaluation. However, understanding within-content engagement on a web page is not a trivial task mainly because of two reasons: (1) engagement is subjective and different users may exhibit different behavioural patterns; (2) existing proxies of user engagement (e.g., clicks, dwell time) suffer from certain caveats, such as the well-known position bias, and are not as effective in discriminating between useful and non-useful components. In this paper, we conduct a crowdsourcing study and examine how users engage with a prominent web search engine component such as the knowledge module (KM) display. To this end, we collect and analyse more than 115k mouse cursor positions from 300 users, who perform a series of search tasks. Furthermore, we engineer a large number of meta-features which we use to predict different proxies of user engagement, including attention and usefulness. In our experiments, we demonstrate that our approach is able to predict more accurately different levels of user engagement and outperform existing baselines.
Simulation of Interaction: A Tutorial on Modelling and Simulating User Interaction and Search Behaviour
Authors: Leif Azzopardi
Venue: SIGIR
Year: 2016

Search is an inherently interactive, non-deterministic and user-dependent process. This means that there are many different possible sequences of interactions which could be taken (some ending in success and others ending in failure). Simulation provides a low cost, repeatable and reproducible way to explore a large range of different possibilities. This makes simulation very appealing, but it also requires care and consideration in developing, implementing and instantiating models of user behaviour for the purposes of experimentation.
Improved Caching Techniques for Large-Scale Image Hosting Services
Authors: Xiao Bai, Berkant Barla Cambazoglu, Archie Russell
Venue: SIGIR
Year: 2016

Commercial image serving systems, such as Flickr and Facebook, rely on large image caches to avoid the retrieval of requested images from the costly backend image store, as much as possible. Such systems serve the same image in different resolutions and, thus, in different sizes to different clients, depending on the properties of the clients' devices. The requested resolutions of images can be cached individually, as in the traditional caches, reducing the backend workload. However, a potentially better approach is to store relatively high-resolution images in the cache and resize them during the retrieval to obtain lower-resolution images. Having this kind of on-the-fly image resizing capability enables image serving systems to deploy more sophisticated caching policies and improve their serving performance further. In this paper, we formalize the static caching problem in image serving systems which provide on-the-fly image resizing functionality in their edge caches or regional caches. We propose two gain-based caching policies that construct a static, fixed-capacity cache to reduce the average serving time of images. The basic idea in the proposed policies is to identify the best resolution(s) of images to be cached so that the average serving time for future image retrieval requests is reduced. We conduct extensive experiments using real-life data access logs obtained from Flickr. We show that one of the proposed caching policies reduces the average response time of the service by up to 4.2% with respect to the best-performing baseline that mainly relies on the access frequency information to make the caching decisions. This improvement implies about 25% reduction in cache size under similar serving time constraints.
Ingrams: A Neuropsychological Explanation For Why People Search
Authors: Peter Bailey, Nick Craswell
Venue: SIGIR
Year: 2016

Why do people start a search? Why do they stop? Why do they do what they do in-between? Our goal in this paper is to provide a simple yet general explanation for these acts that has its basis in neuropsychology and observed user behavior. We coin the term "ingram", as an information counterpart to Richard Semon's ?engram? or "memory trace". People search to create ingrams. People stop searching because they have created sufficient ingrams, or given up. We describe these acts through a pair of user models and use it to explain various user behaviors in search activity. Understanding people?s search acts in terms of ingrams may help us predict or model the interaction of people?s information needs, the queries they issue, and the information they consume. If we could observe certain decision-making acts within these activities, we might also gain new insight into the relationships between textual information and knowledge representation.
R-Susceptibility: An IR-Centric Approach to Assessing Privacy Risks for Users in Online Communities
Authors: Joanna Asia Biega, Krishna P. Gummadi, Ida Mele, Dragan Milchevski, Christos Tryfonopoulos, Gerhard Weikum
Venue: SIGIR
Year: 2016

Privacy of Internet users is at stake because they expose personal information in posts created in online communities, in search queries, and other activities. An adversary that monitors a community may identify the users with the most sensitive properties and utilize this knowledge against them (e.g., by adjusting the pricing of goods or targeting ads of sensitive nature). Existing privacy models for structured data are inadequate to capture privacy risks from user posts.
Utilizing Focused Relevance Feedback
Authors: Elinor Brondwine, Anna Shtok, Oren Kurland
Venue: SIGIR
Year: 2016

We present a novel study of ad hoc retrieval methods utilizing document-level relevance feedback and/or focused relevance feedback; namely, passages marked as (non-)relevant. The first method uses a novel mixture model that integrates relevant and non-relevant information at the language model level. The second method fuses retrieval scores produced by using relevant and non-relevant information separately. Empirical exploration attests to the merits of our methods, and sheds light on the effectiveness of using and integrating relevance feedback for textual units of varying granularities.
Burst Detection in Social Media Streams for Tracking Interest Profiles in Real Time
Authors: Cody Buntain, Jimmy Lin
Venue: SIGIR
Year: 2016

This work presents RTTBurst, an end-to-end system for ingesting descriptions of user interest profiles and discovering new and relevant tweets based on those interest profiles using a simple model for identifying bursts in token usage. Our approach differs from standard retrieval-based techniques in that it primarily focuses on identifying noteworthy moments in the tweet stream, and ?summarizes? those moments using selected tweets. We lay out the architecture of RTTBurst, our participation in and performance at the TREC 2015 Microblog track, and a method for combining and potentially improving existing TREC systems. Official results and post hoc experiments show that our simple targeted burst detection technique is competitive with existing systems. Furthermore, we demonstrate that our burst detection mechanism can be used to improve the performance of other systems for the same task.
Time-Quality Trade-offs in Search
Authors: Ryan Burton
Venue: SIGIR
Year: 2016

In this paper, I propose a research agenda surrounding the notion of slow search, where retrieval speed may be traded for improvements in result quality. This time-quality trade- off leads to a number of implications in the areas of human- computer interaction and information retrieval algorithms, and I plan to explore this space along various dimensions, including adjustments in user behavior when exposed to new search paradigms, investigating the utility a user perceives when given the option to use slow search in addition to traditional search, and examining different notions of 'quality'.