OpenRaiser/Intern-Atlas huggingface dataset:
Learning latent friendship propagation networks with interest awareness for link prediction
Authors: Jun Zhang, Chaokun Wang, Philip S. Yu, Jianmin Wang
Venue: SIGIR
Year: 2013

It's well known that the transitivity of friendship is a popular sociological principle in social networks. However, it's still unknown that to what extent people's friend-making behaviors follow this principle and to what extent it can benefit the link prediction task. In this paper, we try to adopt this sociological principle to explain the evolution of networks and study the latent friendship propagation. Unlike traditional link prediction approaches, we model link formation as results of individuals' friend-making behaviors combined with personal interests. We propose the Latent Friendship Propagation Network (LFPN), which depicts the evolution progress of one's egocentric network and reveals future growth potentials driven by the transitivity of friendship based on personal interests. We model individuals' social behaviors using the Latent Friendship Propagation Model (LFPM), a probabilistic generative model from which the LFPN can be learned effectively. To evaluate the power of the friendship propagation in link prediction, we design LFPN-RW which models the friend-making behavior as a random walk upon the LFPN naturally and captures the co-influence effect of the friend circles as well as personal interests to provide more accurate prediction. Experimental results on real-world datasets show that LFPN-RW outperforms the state-of-the-art approaches. This convinces that the transitivity of friendship actually plays important roles in the evolution of social networks.
An adaptive evidence weighting method for medical record search
Authors: Dongqing Zhu, Ben Carterette
Venue: SIGIR
Year: 2013

In this paper, we present a medical record search system which is useful for identifying cohorts required in clinical studies. In particular, we propose a query-adaptive weighting method that can dynamically aggregate and score evidence in multiple medical reports (from different hospital departments or from different tests within the same department) of a patient. Furthermore, we explore several informative features for learning our retrieval model.
Fresh BrowseRank
Authors: Maxim Zhukovskiy, Andrei Khropov, Gleb Gusev, Pavel Serdyukov
Venue: SIGIR
Year: 2013

In the last years, a lot of attention was attracted by the problem of page authority computation based on user browsing behavior. However, the proposed methods have a number of limitations. In particular, they run on a single snapshot of a user browsing graph ignoring substantially dynamic nature of user browsing activity, which makes such methods recency unaware. This paper proposes a new method for computing page importance, referred to as Fresh BrowseRank. The score of a page by our algorithm equals to the weight in a stationary distribution of a flexible random walk, which is controlled by recency-sensitive weights of vertices and edges. Our method generalizes some previous approaches, provides better capability for capturing the dynamics of the Web and users behavior, and overcomes essential limitations of BrowseRank. The experimental results demonstrate that our method enables to achieve more relevant and fresh ranking results than the classic BrowseRank.
On peculiarities of positional effects in sponsored search
Authors: Vyacheslav Alipov, Valery A. Topinsky, Ilya Trofimov
Venue: SIGIR
Year: 2014

Click logs provide a unique and highly valuable source of human judgments on ads' relevance. However, clicks are heavily biased by lots of factors. Two main factors that are widely acknowledged to be the most influential ones are neighboring ads and presentation order. The latter is referred to as positional effect. A popular practice to recover the ads quality cleaned from positional bias is to adopt click models based on examination or cascade hypothesis originally developed for organic search. In this paper we show the strong evidence that this practice is far from perfection when considering the top ads block on a search engine result page (SERP). We show that cascade hypothesis is the most questionable one because of important differences between organic and sponsored search results that may encourage users to analyze the whole ads-block before clicking. Additionally, we design a testing setup for an unbiased evaluation of click model prediction accuracy.
A general account of effectiveness metrics for information tasks: retrieval, filtering, and clustering
Authors: Enrique Amigó, Julio Gonzalo, Stefano Mizzaro
Venue: SIGIR
Year: 2014

In this tutorial we will present, review, and compare the most popular evaluation metrics for some of the most salient information related tasks, covering: (i) Information Retrieval, (ii) Clustering, and (iii) Filtering. The tutorial will make a special emphasis on the specification of constraints for suitable metrics in each of the three tasks, and on the systematic comparison of metrics according to such constraints. The last part of the tutorial will investigate the challenge of combining and weighting metrics.
Modeling evolution of a social network using temporalgraph kernels
Authors: Akash Anil, Niladri Sett, Sanasam Ranbir Singh
Venue: SIGIR
Year: 2014

Majority of the studies on modeling the evolution of a social network using spectral graph kernels do not consider temporal effects while estimating the kernel parameters. As a result, such kernels fail to capture structural properties of the evolution over the time. In this paper, we propose temporal spectral graph kernels of four popular graph kernels namely path counting, triangle closing, exponential and neumann. Their responses in predicting future growth of the network have been investigated in detail, using two large datasets namely Facebook and DBLP. It is evident from various experimental setups that the proposed temporal spectral graph kernels outperform all of their non-temporal counterparts in predicting future growth of the networks.
An annotation similarity model in passage ranking for historical fact validation
Authors: Jun Araki, Jamie Callan
Venue: SIGIR
Year: 2014

State-of-the-art question answering (QA) systems employ passage retrieval based on bag-of-words similarity models with respect to a query and a passage. We propose a combination of a traditional bag-of-words similarity model and an annotation similarity model to improve passage ranking. The proposed annotation similarity model is generic enough to process annotations of arbitrary types. Historical fact validation is a subtask to determine whether a given sentence tells us historically correct information, which is important for a QA task on world history. Experimental results show that the combined model gains up to 7.7% and 4.2% improvements in historical fact validation in terms of precision at rank 1 and mean reciprocal rank, respectively.
Impact of response latency on user behavior in web search
Authors: Ioannis Arapakis, Xiao Bai, Berkant Barla Cambazoglu
Venue: SIGIR
Year: 2014

Traditionally, the efficiency and effectiveness of search systems have both been of great interest to the information retrieval community. However, an in-depth analysis on the interplay between the response latency of web search systems and users' search experience has been missing so far. In order to fill this gap, we conduct two separate studies aiming to reveal how response latency affects the user behavior in web search. First, we conduct a controlled user study trying to understand how users perceive the response latency of a search system and how sensitive they are to increasing delays in response. This study reveals that, when artificial delays are introduced into the response, the users of a fast search system are more likely to notice these delays than the users of a slow search system. The introduced delays become noticeable by the users once they exceed a certain threshold value. Second, we perform an analysis using a large-scale query log obtained from Yahoo web search to observe the potential impact of increasing response latency on the click behavior of users. This analysis demonstrates that latency has an impact on the click behavior of users to some extent. In particular, given two content-wise identical search result pages, we show that the users are more likely to perform clicks on the result page that is served with lower latency.
Examining collaborative query reformulation: a case of travel information searching
Authors: Abu Shamim Mohammad Arif, Jia Tina Du, Ivan Lee
Venue: SIGIR
Year: 2014

Users often reformulate or modify their queries when they engage in searching information particularly when the search task is complex and exploratory. This paper investigates query reformulation behavior in collaborative tourism information searching on the Web. A user study was conducted with 17 pairs of participants and each pair worked as a team collaboratively on an exploratory travel search task in two scenarios. We analyzed users' collaborative query (CQ) reformulation behavior in two dimensions: firstly, CQ reformulation strategies; and secondly, the effect of individual queries and chat logs on CQ reformulation. The findings show that individual queries and chat logs were two major sources of query terms in CQ reformulation. The statistical results demonstrate the significant effect of individual queries on CQ reformulation. We also found that five operations were performed to reformulate the CQs, namely: addition, modification, reordering, addition and modification, and addition and reordering. These findings have implications for the design of query suggestions that could be offered to users during searches using collaborative search tools.
Evaluating mobile web search performance by taking good abandonment into account
Authors: Olga Arkhipova, Lidia Grauer
Venue: SIGIR
Year: 2014

Usage of mobile devices for Web search grows rapidly in recent years. The common tendency is that users want to receive information immediately results in incorporating rich snippets and vertical results into search engine result pages (SERPs) and in increasing of good abandonment. This article provides an offline metric for quality evaluation of mobile Web search, which takes good abandonment rate into consideration. The metric is the DBN click model that allows the probability to be satisfied directly on the SERP. The model parameters are estimated from the mobile search logs of a controlled experiment. The new metric outperforms traditional ERR metric in terms of the validation dataset built using a SERP degradation technique.
The effect of expanding relevance judgements with duplicates
Authors: Gaurav Baruah, Adam Roegiest, Mark D. Smucker
Venue: SIGIR
Year: 2014

We examine the effects of expanding a judged set of sentences with their duplicates from a corpus. Including new sentences that are exact duplicates of the previously judged sentences may allow for better estimation of performance metrics and enhance the reusability of a test collection. We perform experiments in context of the Temporal Summarization Track at TREC 2013. We find that adding duplicate sentences to the judged set does not significantly affect relative system performance. However, we do find statistically significant changes in the performance of nearly half the systems that participated in the Track. We recommend adding exact duplicate sentences to the set of relevance judgements in order to obtain a more accurate estimate of system performance.
Semantic full-text search with broccoli
Authors: Hannah Bast, Florian Bäurle, Björn Buchhold, Elmar Haußmann
Venue: SIGIR
Year: 2014

We combine search in triple stores with full-text search into what we call \emph{semantic full-text search}. We provide a fully functional web application that allows the incremental construction of complex queries on the English Wikipedia combined with the facts from Freebase. The user is guided by context-sensitive suggestions of matching words, instances, classes, and relations after each keystroke. We also provide a powerful API, which may be used for research tasks or as a back end, e.g., for a question answering system. Our web application and public API are available under \url{http://broccoli.cs.uni-freiburg.de}.
Predicting trending messages and diffusion participants in microblogging network
Authors: Jingwen Bian, Yang Yang, Tat-Seng Chua
Venue: SIGIR
Year: 2014

Microblogging services have emerged as an essential way to strengthen the communications among individuals. One of the most important features of microblog over traditional social networks is the extensive proliferation in information diffusion. As the outbreak of information diffusion often brings in valuable opportunities or devastating effects, it will be beneficial if a mechanism can be provided to predict whether a piece of information will become viral, and which part of the network will participate in propagating this information. In this work, we define three types of influences, namely, interest-oriented influence, social-oriented influence, and epidemic-oriented influence, that will affect a user's decision on whether to perform a diffusion action. We propose a diffusion-targeted influence model to differentiate and quantify various types of influence. Further we model the problem of diffusion prediction by factorizing a user's intention to transmit a microblog into these influences. The learned prediction model is then used to predict the future diffusion state of any new microblog. We conduct experiments on a real-world microblogging dataset to evaluate our method, and the results demonstrate the superiority of the proposed framework as compared to the state-of-the-art approaches.
Web page segmentation with structured prediction and its application in web page classification
Authors: Lidong Bing, Rui Guo, Wai Lam, Zheng-Yu Niu, Haifeng Wang
Venue: SIGIR
Year: 2014

We propose a framework which can perform Web page segmentation with a structured prediction approach. It formulates the segmentation task as a structured labeling problem on a transformed Web page segmentation graph (WPS-graph). WPS-graph models the candidate segmentation boundaries of a page and the dependency relation among the adjacent segmentation boundaries. Each labeling scheme on the WPS-graph corresponds to a possible segmentation of the page. The task of finding the optimal labeling of the WPS-graph is transformed into a binary Integer Linear Programming problem, which considers the entire WPS-graph as a whole to conduct structured prediction. A learning algorithm based on the structured output Support Vector Machine framework is developed to determine the feature weights, which is capable to consider the inter-dependency among candidate segmentation boundaries. Furthermore, we investigate its efficacy in supporting the development of automatic Web page classification.
Personalized document re-ranking based on Bayesian probabilistic matrix factorization
Authors: Fei Cai, Shangsong Liang, Maarten de Rijke
Venue: SIGIR
Year: 2014

A query considered in isolation provides limited information about the searcher's interest. Previous work has considered various types of user behavior, e.g., clicks and dwell time, to obtain a better understanding of the user's intent. We consider the searcher's search and page view history. Using search logs from a commercial search engine, we (i) investigate the impact of features derived from user behavior on reranking a generic ranked list; (ii) optimally integrate the contributions of user behavior and candidate documents by learning their relative importance per query based on similar users. We use dwell time on clicked URLs when estimating the relevance of documents for a query, and perform Bayesian Probabilistic Matrix Factorization as smoothing to predict the relevance. Considering user behavior achieves better rankings than non-personalized rankings. Aggregation of user behavior and query-document features with a user-dependent adaptive weight outperforms combinations with a fixed uniform value.
Scalability and efficiency challenges in large-scale web search engines
Authors: Berkant Barla Cambazoglu, Ricardo Baeza-Yates
Venue: SIGIR
Year: 2014

Large-scale web search engines rely on massive compute infrastructures to be able to cope with the continuous growth of the Web and their user bases. In such search engines, achieving scalability and efficiency requires making careful architectural design choices while devising algorithmic performance optimizations. Unfortunately, most details about the internal functioning of commercial web search engines remain undisclosed due to their financial value and the high level of competition in the search market. The main objective of this tutorial is to provide an overview of the fundamental scalability and efficiency challenges in commercial web search engines, bridging the existing gap between the industry and academia.
Incorporating query-specific feedback into learning-to-rank models
Authors: Ethem F. Can, W. Bruce Croft, R. Manmatha
Venue: SIGIR
Year: 2014

Relevance feedback has been shown to improve retrieval for a broad range of retrieval models. It is the most common way of adapting a retrieval model for a specific query. In this work, we expand this common way by focusing on an approach that enables us to do query-specific modification of a retrieval model for learning-to-rank problems. Our approach is based on using feedback documents in two ways: 1) to improve the retrieval model directly and 2) to identify a subset of training queries that are more predictive than others. Experiments with the Gov2 collection show that this approach can obtain statistically significant improvements over two baselines; learning-to-rank (SVM-rank) with no feedback and learning-to-rank with standard relevance feedback.
Latent community discovery through enterprise user search query modeling
Authors: Kevin M. Carter, Rajmonda Sulo Caceres, Ben Priest
Venue: SIGIR
Year: 2014

Enterprise computer networks are filled with users performing a variety of tasks, ranging from business-critical tasks to personal interest browsing. Due to this multi-modal distribution of behaviors, it is non-trivial to automatically discern which behaviors are business-relevant and which are not. Additionally, it is difficult to infer communities of interest within the enterprise, even given an organizational mapping. In this work, we present a two-step framework for classifying user behavior within an enterprise in a data-driven way. As a first step, we use a latent topic model on active search queries to identify types of behaviors and topics of interest associated with a given user. We then leverage the information about user's assigned role within the organization to extract relevant topics which are most reflective of self-organizing communities of interest. We demonstrate that our framework is able to identify rich communities of interest that are better representations of how users interact and assemble in an enterprise setting.
Statistical significance testing in information retrieval: theory and practice
Authors: Ben Carterette
Venue: SIGIR
Year: 2014

The past 20 years have seen a great improvement in the rigor of information retrieval experimentation, due primarily to two factors: high-quality, public, portable test collections such as those produced by TREC (the Text REtrieval Con- ference [2]), and the increased practice of statistical hypothesis testing to determine whether measured improvements can be ascribed to something other than random chance. Together these create a very useful standard for reviewers, program committees, and journal editors; work in information retrieval (IR) increasingly cannot be published unless it has been evaluated using a well-constructed test collection and shown to produce a statistically significant improvement over a good baseline.
On correlation of absence time and search effectiveness
Authors: Sunandan Chakraborty, Filip Radlinski, Milad Shokouhi, Paul Baecke
Venue: SIGIR
Year: 2014

Online search evaluation metrics are typically derived based on implicit feedback from the users. For instance, computing the number of page clicks, number of queries, or dwell time on a search result. In a recent paper, Dupret and Lalmas introduced a new metric called absence time, which uses the time interval between successive sessions of users to measure their satisfaction with the system. They evaluated this metric on a version of Yahoo! Answers. In this paper, we investigate the effectiveness of absence time in evaluating new features in a web search engine, such as new ranking algorithm or a new user interface. We measured the variation of absence time to the effects of 21 experiments performed on a search engine. Our findings show that the outcomes of absence time agreed with the judgement of human experts performing a thorough analysis of a wide range of online and offline metrics in 14 out of these 21 cases.
Inferring topic-dependent influence roles of Twitter users
Authors: Chengyao Chen, Dehong Gao, Wenjie Li, Yuexian Hou
Venue: SIGIR
Year: 2014

Twitter, as one of the most popular social media platforms, provides a convenient way for people to communicate and interact with each other. It has been well recognized that influence exists during users' interactions. Some pioneer studies on finding influential users have been reported in the literature, but they do not distinguish different influence roles, which are of great value for various marketing purposes. In this paper, we move a step forward trying to further distinguish influence roles of Twitter users in a certain topic. By defining three views of features relating to topic, sentiment and popularity respectively, we propose a Multi-view Influence Role Clustering (MIRC) algorithm to group Twitter users into five categories. Experimental results show the effectiveness of the proposed approach in inferring influence roles.
Who is the barbecue king of texas?: a geo-spatial approach to finding local experts on twitter
Authors: Zhiyuan Cheng, James Caverlee, Himanshu Barthwal, Vandana Bachani
Venue: SIGIR
Year: 2014

This paper addresses the problem of identifying local experts in social media systems like Twitter. Local experts -- in contrast to general topic experts -- have specialized knowledge focused around a particular location, and are important for many applications including answering local information needs and interacting with community experts. And yet identifying these experts is difficult. Hence in this paper, we propose a geo-spatial-driven approach for identifying local experts that leverages the fine-grained GPS coordinates of millions of Twitter users. We propose a local expertise framework that integrates both users' topical expertise and their local authority. Concretely, we estimate a user's local authority via a novel spatial proximity expertise approach that leverages over 15 million geo-tagged Twitter lists. We estimate a user's topical expertise based on expertise propagation over 600 million geo-tagged social connections on Twitter. We evaluate the proposed approach across 56 queries coupled with over 11,000 individual judgments from Amazon Mechanical Turk. We find significant improvement over both general (non-local) expert approaches and comparable local expert finding approaches.
Just-for-me: an adaptive personalization system for location-aware social music recommendation
Authors: Zhiyong Cheng, Jialie Shen, Tao Mei
Venue: SIGIR
Year: 2014

In recent years, location-aware music recommendation is increasing in popularity, as more and more users consume music on the move. In this demonstration, we present an intelligent system, called Just-for-Me, to facilitate accurate music recommendation based on where user presents. Our system is developed based on a novel probabilistic generative model, which can effectively integrate the location contexts and global music popularity trends. This approach allows us to gain more comprehensive modeling on user preference and thus significantly enhances the music recommendation performance.