OpenRaiser/Intern-Atlas huggingface dataset:
QUADS: question answering for decision support
Authors: Zi Yang, Ying Li, James Cai, Eric Nyberg
Venue: SIGIR
Year: 2014

As the scale of available on-line data grows ever larger, individuals and businesses must cope with increasing complexity in decision-making processes which utilize large volumes of unstructured, semi-structured and/or structured data to satisfy multiple, interrelated information needs which contribute to an overall decision. Traditional decision support systems (DSSs) have been developed to address this need, but such systems are typically expensive to build, and are purpose-built for a particular decision-making scenario, making them difficult to extend or adapt to new decision scenarios. In this paper, we propose a novel decision representation which allows decision makers to formulate and organize natural language questions or assertions into an analytic hierarchy, which can be evaluated as part of an ad hoc decision process or as a documented, repeatable analytic process. We then introduce a new decision support framework, QUADS, which takes advantage of automatic question answering (QA) technologies to automatically understand and process a decision representation, producing a final decision by gathering and weighting answers to individual questions using a Bayesian learning and inference process. An open source framework implementation is presented and applied to two real world applications: target validation, a fundamental decision-making task for the pharmaceutical industry, and product recommendation from review texts, an everyday decision-making situation faced by on-line consumers. In both applications, we implemented and compared a number of decision synthesis algorithms, and present experimental results which demonstrate the performance of the QUADS approach versus other baseline approaches.
Dynamic information retrieval modeling
Authors: Hui Yang, Marc Sloan, Jun Wang
Venue: SIGIR
Year: 2014

Dynamic aspects of Information Retrieval (IR), including changes found in data, users and systems, are increasingly being utilized in search engines and information filtering systems. Existing IR techniques are limited in their ability to optimize over changes, learn with minimal computational footprint and be responsive and adaptive. The objective of this tutorial is to provide a comprehensive and up-to-date introduction to Dynamic Information Retrieval Modeling, the statistical modeling of IR systems that can adapt to change. It will cover techniques ranging from classic relevance feedback to the latest applications of partially observable Markov decision processes (POMDPs) and a handful of useful algorithms and tools for solving IR problems incorporating dynamics.
Exploring recommendations in internet of things
Authors: Lina Yao, Quan Z. Sheng, Anne H. H. Ngu, Helen Ashman, Xue Li
Venue: SIGIR
Year: 2014

With recent advances in radio-frequency identification (RFID), wireless sensor networks, and Web-based services, physical things are becoming an integral part of the emerging ubiquitous Web. In this paper, we focus on the things recommendation problem in Internet of Things (IoT). In particular, we propose a unified probabilistic based framework by fusing information across relationships between users (i.e., users'social network) and things (i.e., things correlations) to make more accurate recommendations. The proposed approach not only inherits the advantages of the matrix factorization, but also exploits the merits of social relationships and thing-thing correlations. We validate our approach based on an Internet of Things platform and the experimental results demonstrate its feasibility and effectiveness.
A simple term frequency transformation model for effective pseudo relevance feedback
Authors: Zheng Ye, Jimmy Xiangji Huang
Venue: SIGIR
Year: 2014

Pseudo Relevance Feedback is an effective technique to improve the performance of ad-hoc information retrieval. Traditionally, the expansion terms are extracted either according to the term distributions in the feedback documents; or according to both the term distributions in the feedback documents and in the whole document collection. However, most of the existing models employ a single term frequency normalization mechanism or criteria that cannot take into account various aspects of a term's saliency in the feedback documents. In this paper, we propose a simple and heuristic, but effective model, in which three term frequency transformation techniques are integrated to capture the saliency of a candidate term associated with the original query terms in the feedback documents. Through evaluations and comparisons on six TREC collections, we show that our proposed model is effective and generally superior to the recent progress of relevance feedback models.
Analyzing bias in CQA-based expert finding test sets
Authors: Reyyan Yeniterzi, Jamie Callan
Venue: SIGIR
Year: 2014

Data retrieved from community question answering (CQA) sites, such as content and users' assessments of content, is commonly used for expertise estimation related tasks. One such task, in which the received votes are directly used as graded relevance assessment values, is ranking replies of a question. Even though these available assessments values are very practical for evaluation purposes, they may not always reflect the correct assessment value of the content, due to the possible temporal or presentation bias introduced by the CQA system during voting process. This paper analyzes a very commonly used CQA data collection in terms of these introduced biases and their effects on the experimental evaluation of approaches. A more bias free test set construction approach, which has correlated results with the manual assessments, is also proposed in this paper.
Sig-SR: SimRank search over singular graphs
Authors: Weiren Yu, Julie A. McCann
Venue: SIGIR
Year: 2014

SimRank is an attractive structural-context measure of similarity between two objects in a graph. It recursively follows the intuition that "two objects are similar if they are referenced by similar objects". The best known matrix-based method [1] for calculating SimRank, however, implies an assumption that the graph is non-singular, its adjacency matrix is invertible. In reality, non-singular graphs are very rare; such an assumption in [1] is too restrictive in practice. In this paper, we provide a treatment of [1], by supporting similarity assessment on non-invertible adjacency matrices. Assume that a singular graph G has n nodes, with r(<n) being the rank of its adjacency matrix.(1) We show that SimRank matrix S on G has an elegant structure: S can be represented as a rank r matrix plus a scaled identity matrix.(2) By virtue of this, an efficient algorithm over singular graphs, InvSR, is proposed for calculating all-pairs SimRank in O(r(n2+Kr2)) time for K iterations. In contrast, the only known matrix-based algorithm that supports singular graphs [1] needs O(r4n2) time. The experimental results on real and synthetic datasets demonstrate the superiority of InvSR on singular graphs against its baselines.
Hashing with List-Wise learning to rank
Authors: Zhou Yu, Fei Wu, Yin Zhang, Siliang Tang, Jian Shao, Yueting Zhuang
Venue: SIGIR
Year: 2014

Hashing techniques have been extensively investigated to boost similarity search for large-scale high-dimensional data. Most of the existing approaches formulate the their objective as a pair-wise similarity-preserving problem. In this paper, we consider the hashing problem from the perspective of optimizing a list-wise learning to rank problem and propose an approach called List-Wise supervised Hashing (LWH). In LWH, the hash functions are optimized by employing structural SVM in order to explicitly minimize the ranking loss of the whole list-wise permutations instead of merely the point-wise or pair-wise supervision. We evaluate the performance of LWH on two real-world data sets. Experimental results demonstrate that our method obtains a significant improvement over the state-of-the-art hashing approaches due to both structural large margin and list-wise ranking pursuing in a supervised manner.
NicePic!: a system for extracting attractive photos from flickr streams
Authors: Sergej Zerr, Stefan Siersdorfer, José San Pedro, Jonathon S. Hare, Xiaofei Zhu
Venue: SIGIR
Year: 2014

A large number of images are continuously uploaded to popular photo sharing websites and online social communities. In this demonstration we show a novel application which automatically classifies images in a live photo stream according to their attractiveness for the community, based on a number of visual and textual features. The system effectively introduces an additional facet to browse and explore photo collections by highlighting the most attractive photographs and demoting the least attractive.
Graph-based large scale RDF data compression
Authors: Wei Emma Zhang
Venue: SIGIR
Year: 2014

We propose a two-stage lossless compression approach on large scale RDF data. Our approach exploits both Representation Compression and Component Compression techniques to support query and dynamic operations directly on the compressed data.
Processing spatial keyword query as a top-k aggregation query
Authors: Dongxiang Zhang, Chee-Yong Chan, Kian-Lee Tan
Venue: SIGIR
Year: 2014

We examine the spatial keyword search problem to retrieve objects of interest that are ranked based on both their spatial proximity to the query location as well as the textual relevance of the object’s keywords. Existing solutions for the problem are based on either using a combination of textual and spatial indexes or using specialized hybrid indexes that integrate the indexing of both textual and spatial attribute values. In this paper, we propose a new approach that is based on modeling the problem as a top-k aggregation problem which enables the design of a scalable and efficient solution that is based on the ubiquitous inverted list index. Our performance study demonstrates that our approach outperforms the state-of-theart hybrid methods by a wide margin.
PatentLine: analyzing technology evolution on multi-view patent graphs
Authors: Longhui Zhang, Lei Li, Tao Li, Qi Zhang
Venue: SIGIR
Year: 2014

The fast growth of technologies has driven the advancement of our society. It is often necessary to quickly grab the evolution of technologies in order to better understand the technology trend. The availability of huge volumes of granted patent documents provides a reasonable basis for analyzing technology evolution. In this paper, we propose a unified framework, named PatentLine, to generate a technology evolution tree for a given topic or a classification code related to granted patents. The framework integrates different types of patent information, including patent content, citations of patents, temporal relations, etc., and provides a concise yet comprehensive evolution summary. The generated summary enables a variety of patent-related analyses such as identifying relevant prior art and detecting technology gap. A case study on a collection of US patents demonstrates the efficacy of our proposed framework.
A POMDP model for content-free document re-ranking
Authors: Sicong Zhang, Jiyun Luo, Hui Yang
Venue: SIGIR
Year: 2014

Log-based document re-ranking is a special form of session search. The task re-ranks documents from Search Engine Results Page (SERP) according to the search logs, in which both the search activities from other users and personalized query log for a user are available. The purpose of re-ranking is to provide the user with a new and better ordering of the initial retrieved documents. We test the system on the WSCD 2014 dataset, in which the actual content of the queries and documents are not available due to privacy concerns. The challenge is to perform effective re-ranking purely based on user behaviors, such as clicks and query reformulations rather than document content. In this paper, we propose to model log-based document re-ranking as a Partially Observable Markov Decision Process (POMDP). Experiments on the document re-ranking task show that our approach is effective and outperforms the baseline rankings provided by a commercial search engine.
Addressing cold start in recommender systems: a semi-supervised co-training algorithm
Authors: Mi Zhang, Jie Tang, Xuchen Zhang, Xiangyang Xue
Venue: SIGIR
Year: 2014

Cold start is one of the most challenging problems in recommender systems. In this paper we tackle the cold-start problem by proposing a context-aware semi-supervised co-training method named CSEL. Specifically, we use a factorization model to capture fine-grained user-item context. Then, in order to build a model that is able to boost the recommendation performance by leveraging the context, we propose a semi-supervised ensemble learning algorithm. The algorithm constructs different (weak) prediction models using examples with different contexts and then employs the co-training strategy to allow each (weak) prediction model to learn from the other prediction models. The method has several distinguished advantages over the standard recommendation methods for addressing the cold-start problem. First, it defines a fine-grained context that is more accurate for modeling the user-item preference. Second, the method can naturally support supervised learning and semi-supervised learning, which provides a flexible way to incorporate the unlabeled data.
Supervised hashing with latent factor models
Authors: Peichao Zhang, Wei Zhang, Wu-Jun Li, Minyi Guo
Venue: SIGIR
Year: 2014

Due to its low storage cost and fast query speed, hashing has been widely adopted for approximate nearest neighbor search in large-scale datasets. Traditional hashing methods try to learn the hash codes in an unsupervised way where the metric (Euclidean) structure of the training data is preserved. Very recently, supervised hashing methods, which try to preserve the semantic structure constructed from the semantic labels of the training points, have exhibited higher accuracy than unsupervised methods. In this paper, we propose a novel supervised hashing method, called latent factor hashing(LFH), to learn similarity-preserving binary codes based on latent factor models. An algorithm with convergence guarantee is proposed to learn the parameters of LFH. Furthermore, a linear-time variant with stochastic learning is proposed for training LFH on large-scale datasets. Experimental results on two large datasets with semantic labels show that LFH can achieve superior accuracy than state-of-the-art methods with comparable training time.
An enhanced context-sensitive proximity model for probabilistic information retrieval
Authors: Jiashu Zhao, Jimmy Xiangji Huang
Venue: SIGIR
Year: 2014

We propose to enhance proximity-based probabilistic retrieval models with more contextual information. A term pair with higher contextual relevance of term proximity is assigned a higher weight. Several measures are proposed to estimate the contextual relevance of term proximity. We assume the top ranked documents from a basic weighting model are more relevant to the query, and calculate the contextual relevance of term proximity using the top ranked documents. We propose a context-sensitive proximity model, and the experimental results on standard TREC data sets show the effectiveness of our proposed model.
Detection of abnormal profiles on group attacks in recommender systems
Authors: Wei Zhou, Yun Sing Koh, Junhao Wen, Shafiq Alam, Gillian Dobbie
Venue: SIGIR
Year: 2014

Recommender systems using Collaborative Filtering techniques are capable of make personalized predictions. However, these systems are highly vulnerable to profile injection attacks. Group attacks are attacks that target a group of items instead of one, and there are common attributes among these items. Such profiles will have a good probability of being similar to a large number of user profiles, making them hard to detect. We propose a novel technique for identifying group attack profiles which uses an improved metric based on Degree of Similarity with Top Neighbors (DegSim) and Rating Deviation from Mean Agreement (RDMA). We also extend our work with a detailed analysis of target item rating patterns. Experiments show that the combined methods can improve detection rates in user-based recommender systems.
Learning for search result diversification
Authors: Yadong Zhu, Yanyan Lan, Jiafeng Guo, Xueqi Cheng, Shuzi Niu
Venue: SIGIR
Year: 2014

Search result diversification has gained attention as a way to tackle the ambiguous or multi-faceted information needs of users. Most existing methods on this problem utilize a heuristic predefined ranking function, where limited features can be incorporated and extensive tuning is required for different settings. In this paper, we address search result diversification as a learning problem, and introduce a novel relational learning-to-rank approach to formulate the task. However, the definitions of ranking function and loss function for the diversification problem are challenging. In our work, we firstly show that diverse ranking is in general a sequential selection process from both empirical and theoretical aspects. On this basis, we define ranking function as the combination of relevance score and diversity score between the current document and those previously selected, and loss function as the likelihood loss of ground truth based on Plackett-Luce model, which can naturally model the sequential generation of a diverse ranking list. Stochastic gradient descent is then employed to conduct the unconstrained optimization, and the prediction of a diverse ranking list is provided by a sequential selection process based on the learned ranking function. The experimental results on the public TREC datasets demonstrate the effectiveness and robustness of our approach.
An adaptive teleportation random walk model for learning social tag relevance
Authors: Xiaofei Zhu, Wolfgang Nejdl, Mihai Georgescu
Venue: SIGIR
Year: 2014

Social tags are known to be a valuable source of information for image retrieval and organization. However, contrary to the conventional document retrieval, rich tag frequency information in social sharing systems, such as Flickr, is not available, thus we cannot directly use the tag frequency (analogous to the term frequency in a document) to represent the relevance of tags. Many heuristic approaches have been proposed to address this problem, among which the well-known neighbor voting based approaches are the most effective methods. The basic assumption of these methods is that a tag is considered as relevant to the visual content of a target image if this tag is also used to annotate the visual neighbor images of the target image by lots of different users. The main limitation of these approaches is that they treat the voting power of each neighbor image either equally or simply based on its visual similarity. In this paper, we cast the social tag relevance learning problem as an adaptive teleportation random walk process on the voting graph. In particular, we model the relationships among images by constructing a voting graph, and then propose an adaptive teleportation random walk, in which a confidence factor is introduced to control the teleportation probability, on the voting graph. Through this process, direct and indirect relationships among images can be explored to cooperatively estimate the tag relevance. To quantify the performance of our approach, we compare it with state-of-the-art methods on two publicly available datasets (NUS-WIDE and MIR Flickr). The results indicate that our method achieves substantial performance gains on these datasets.
Practical Lessons for Gathering Quality Labels at Scale
Authors: Omar Alonso
Venue: SIGIR
Year: 2015

Information retrieval researchers and engineers use human computation as a mechanism to produce labeled data sets for product development, research and experimentation. To gather useful results, a successful labeling task relies on many different elements: clear instructions, user interface guidelines, representative high-quality datasets, appropriate inter-rater agreement metrics, work quality checks, and channels for worker feedback. Furthermore, designing and implementing tasks that produce and use several thousands or millions of labels is different than conducting small scale research investigations. In this paper we present a perspective for collecting high quality labels with an emphasis on practical problems and scalability. We focus on three main topics: programming crowds, debugging tasks with low agreement, and algorithms for quality control. We show examples from an industrial setting.
Galean: Visualization of Geolocated News Events from Social Media
Authors: Vanessa Peña Araya, Mauricio Quezada, Barbara Poblete
Venue: SIGIR
Year: 2015

Online Social Networks (OSN) have changed the way information is produced and consumed. Organizing and retrieving unstructured data extracted from these platforms is not an easy task. Galean is a visual and interactive tool that aims to help journalists and historians, among others, analyze news events discussed on Twitter. In this tool, news events are visually represented by the very countries from where the news originated, the date when they happened and their impact in the OSN. Galean considers countries as entities, as opposed to mere geographical locations as most of the tools in the state of the art. As a consequence, it allows users to explore and retrieve news not only by their geographical and temporal context, but also by the relationship among countries. With this tool users can search for behavioral patterns of news events and observe how countries are associated in specific events. We expect our work to become a public tool that helps conduct historical analyses of social media news coverage over time.
Search Engine Evaluation based on Search Engine Switching Prediction
Authors: Olga Arkhipova, Lidia Grauer, Igor Kuralenok, Pavel Serdyukov
Venue: SIGIR
Year: 2015

In this paper we present a novel application of the search engine switching prediction model for online evaluation. We propose a new metric pSwitch for A/B-testing, which allows us to evaluate the quality of search engines in different aspects such as the quality of the user interface and the quality of the ranking function. pSwitch is a search session-level metric, which relies on the predicted probability that the session contains a switch to another search engine and reflects the degree of the failure of the session. We demonstrate the effectiveness and validity of pSwitch using A/B-testing experiments with real users of search engine Yandex. We compare our metric with recently proposed SpU (sessions per user) metric and other widely used query-level A/B metrics, such as Abandonment Rate and Time to First Click, which we used as our baseline metrics. We observed that pSwitch metric is more sensitive in comparison with those baseline metrics and also that pSwitch and SpU are more consistent with ground truth, than Abandonment Rate and Time to First Click.