OpenRaiser/Intern-Atlas huggingface dataset:
Incorporating popularity in topic models for social network analysis
Authors: Youngchul Cha, Bin Bi, Chu-Cheng Hsieh, Junghoo Cho
Venue: SIGIR
Year: 2013

Topic models are used to group words in a text dataset into a set of relevant topics. Unfortunately, when a few words frequently appear in a dataset, the topic groups identified by topic models become noisy because these frequent words repeatedly appear in “irrelevant ” topic groups. This noise has not been a serious problem in a text dataset because the frequent words (e.g., the and is) do not have much meaning and have been simply removed before a topic model analysis. However, in a social network dataset we are interested in, they correspond to popular persons (e.g., Barack Obama and Justin Bieber) and cannot be simply removed because most people are interested in them. To solve this “popularity problem”, we explicitly model the popularity of nodes (words) in topic models. For this purpose, we first introduce a notion of a “popularity component” and propose topic model extensions that effectively accommodate the popularity component. We evaluate the effectiveness of our models with a real-world Twitter dataset. Our proposed models achieve significantly lower perplexity (i.e., better prediction power) compared to the state-of-theart baselines. In addition to the popularity problem caused by the nodes with high incoming edge degree, we also investigate the effect of the outgoing edge degree with another topic model extensions. We show that considering outgoing edge degree does not help much in achieving lower perplexity.
Document features predicting assessor disagreement
Authors: Praveen Chandar, William Webber, Ben Carterette
Venue: SIGIR
Year: 2013

The notion of relevance differs between assessors, thus giving rise to assessor disagreement. Although assessor disagreement has been frequently observed, the factors leading to disagreement are still an open problem. In this paper we study the relationship between assessor disagreement and various topic independent factors such as readability and cohesiveness. We build a logistic model using reading level and other simple document features to predict assessor disagreement and rank documents by decreasing probability of disagreement. We compare the predictive power of these document-level features with that of a meta-search feature that aggregates a document's ranking across multiple retrieval runs. Our features are shown to be on a par with the meta-search feature, without requiring a large and diverse set of retrieval runs to calculate. Surprisingly, however, we find that the reading level features are negatively correlated with disagreement, suggesting that they are detecting some other aspect of document content.
Modeling user's receptiveness over time for recommendation
Authors: Wei Chen, Wynne Hsu, Mong-Li Lee
Venue: SIGIR
Year: 2013

Existing recommender systems model user interests and the social influences independently. In reality, user interests may change over time, and as the interests change, new friends may be added while old friends grow apart and the new friendships formed may cause further interests change. This complex interaction requires the joint modeling of user interest and social relationships over time. In this paper, we propose a probabilistic generative model, called Receptiveness over Time Model (RTM), to capture this interaction. We design a Gibbs sampling algorithm to learn the receptiveness and interest distributions among users over time. The results of experiments on a real world dataset demonstrate that RTM-based recommendation outperforms the state-of-the-art recommendation methods. Case studies also show that RTM is able to discover the user interest shift and receptiveness change over time
Tagcloud-based explanation with feedback for recommender systems
Authors: Wei Chen, Wynne Hsu, Mong-Li Lee
Venue: SIGIR
Year: 2013

Personalized recommender systems aim to push only the relevant items and information directly to the users without requiring them to browse through millions of web resources. The challenge of these systems is to achieve a high user acceptance rate on their recommendations. In this paper, we aim to increase the user acceptance of recommendations by providing more intuitive tag-based explanations of why the items are recommended. Tags are used as intermediary entities that not only relate target users to the recommended items but also understand users' intents. Our system also allows tag-based online relevance feedback. Experiment results on the Movielens dataset show that the proposed approach is able to increase the acceptance rate of recommendations and improve user satisfaction.
Question retrieval with user intent
Authors: Long Chen, Dell Zhang, Mark Levene
Venue: SIGIR
Year: 2013

Community Question Answering (CQA) services, such as Yahoo! Answers and WikiAnswers, have become popular with users as one of the central paradigms for satisfying users' information needs. The task of question retrieval in CQA aims to resolve one's query directly by finding the most relevant questions (together with their answers) from an archive of past questions. However, as users can ask any question that they like, a large number of questions in CQA are not about objective (factual) knowledge, but about subjective (sentiment-based) opinions or social interactions. The inhomogeneous nature of CQA leads to reduced performance of standard retrieval models. To address this problem, we present a hybrid approach that blends several language modelling techniques for question retrieval, namely, the classic (query-likelihood) language model, the state-of-the-art translation-based language model, and our proposed intent-based language model. The user intent of each candidate question (objective/subjective/social) is given by a probabilistic classifier which makes use of both textual features and metadata features. Our experiments on two real-world datasets show that our approach can significantly outperform existing ones.
SIGIR 2013 workshop on modeling user behavior for information retrieval evaluation
Authors: Charles L. A. Clarke, Luanne Freund, Mark D. Smucker, Emine Yilmaz
Venue: SIGIR
Year: 2013

The SIGIR 2013 Workshop on Modeling User Behavior for Information Retrieval Evaluation (MUBE 2013) brings together people to discuss existing and new approaches, ways to collaborate, and other ideas and issues involved in improving information retrieval evaluation through the modeling of user behavior.
Diversified relevance feedback
Authors: Matt Crane
Venue: SIGIR
Year: 2013

The need for a search engine to deal with ambiguous queries has been known for a long time (diversification). However, it is only recently that this need has become a focus within information retrieval research. How to respond to indications that a result is relevant to a query (relevance feedback) has also been a long focus of research. When thinking about the results for a query as being clustered by topic, these two areas of information retrieval research appear to be opposed to each other. Interestingly though, they both appear to improve the performance of search engines, raising the question: they can be combined or made to work with each other?
Ranking-oriented nearest-neighbor based method for automatic image annotation
Authors: Chaoran Cui, Jun Ma, Tao Lian, Xiaofang Wang, Zhaochun Ren
Venue: SIGIR
Year: 2013

Automatic image annotation plays a critical role in keyword-based image retrieval systems. Recently, the nearest-neighbor based scheme has been proposed and achieved good performance for image annotation. Given a new image, the scheme is to first find its most similar neighbors from labeled images, and then propagate the keywords associated with the neighbors to it. Many studies focused on designing a suitable distance metric between images so that all labeled images can be ranked by their distance to the given image. However, higher accuracy in distance prediction does not necessarily lead to better ordering of labeled images. In this paper, we propose a ranking-oriented neighbor search mechanism to rank labeled images directly without going through the intermediate step of distance prediction. In particular, a new learning to rank algorithm is developed, which exploits the implicit preference information of labeled images and underlines the accuracy of the top-ranked results. Experiments on two benchmark datasets demonstrate the effectiveness of our approach for image annotation.
Exploiting user feedback to learn to rank answers in q&a forums: a case study with stack overflow
Authors: Daniel Hasan Dalip, Marcos André Gonçalves, Marco Cristo, Pável Calado
Venue: SIGIR
Year: 2013

Collaborative web sites, such as collaborative encyclopedias, blogs, and forums, are characterized by a loose edit control, which allows anyone to freely edit their content. As a consequence, the quality of this content raises much concern. To deal with this, many sites adopt manual quality control mechanisms. However, given their size and change rate, manual assessment strategies do not scale and content that is new or unpopular is seldom reviewed. This has a negative impact on the many services provided, such as ranking and recommendation. To tackle with this problem, we propose a learning to rank (L2R) approach for ranking answers in Q&A forums. In particular, we adopt an approach based on Random Forests and represent query and answer pairs using eight different groups of features. Some of these features are used in the Q&A domain for the first time. Our L2R method was trained to learn the answer rating, based on the feedback users give to answers in Q&A forums. Using the proposed method, we were able (i) to outperform a state of the art baseline with gains of up to 21% in NDCG, a metric used to evaluate rankings; we also conducted a comprehensive study of the features, showing that (ii) review and user features are the most important in the Q&A domain although text features are useful for assessing quality of new answers; and (iii) the best set of new features we proposed was able to yield the best quality rankings.
Efficient query construction for large scale data
Authors: Elena Demidova, Xuan Zhou, Wolfgang Nejdl
Venue: SIGIR
Year: 2013

In recent years, a number of open databases have emerged on the Web, providing Web users with platforms to collaboratively create structured information. As these databases are intended to accommodate heterogeneous information and knowledge, they usually comprise a very large schema and billions of instances. Browsing and searching data on such a scale is not an easy task for a Web user. In this context, interactive query construction offers an intuitive interface for novice users to retrieve information from databases neither requiring any knowledge of structured query languages, nor any prior knowledge of the database schema. However, the existing mechanisms do not scale well on large scale datasets. This paper presents a set of techniques to boost the scalability of interactive query construction, from the perspective of both, user interaction cost and performance. We connect an abstract ontology layer to the database schema to shorten the process of user-computer interaction. We also introduce a search mechanism to enable efficient exploration of query interpretation spaces over large scale data. Extensive experiments show that our approach scales well on Freebase - an open database containing more than 7,000 relational tables in more than 100 domains.
Estimating topical context by diverging from external resources
Authors: Romain Deveaud, Eric SanJuan, Patrice Bellot
Venue: SIGIR
Year: 2013

Improving query understanding is crucial for providing the user with information that suits her needs. To this end, the retrieval system must be able to deal with several sources of knowledge from which it could infer a topical context. The use of external sources of information for improving document retrieval has been extensively studied. Improvements with either structured or large sets of data have been reported. However, in these studies resources are often used separately and rarely combined together. We experiment in this paper a method that discounts documents based on their weighted divergence from a set of external resources. We present an evaluation of the combination of four resources on two standard TREC test collections. Our proposed method significantly outperforms a state-of-the-art Mixture of Relevance Models on one test collection, while no significant differences are detected on the other one.
SIGIR 2013 workshop on time aware information access (#TAIA2013)
Authors: Fernando Diaz, Susan T. Dumais, Miles Efron, Kira Radinsky, Maarten de Rijke, Milad Shokouhi
Venue: SIGIR
Year: 2013

Web content increasingly reflects the current state of the physical and social world, manifested both in traditional news media sources along with user-generated publishing sites such as Twitter, Foursquare, and Facebook. At the same time, web searching increasingly reflects problems grounded in the real world. As a result of this blending of the web with the real world, we observe that the web, both in its composition and use, has incorporated many of the dynamics of the real world. Few of the problems associated with searching dynamic collections are well understood, such as defining time-sensitive relevance, understanding user query behavior over time and understanding why certain web content changes. We believe that, just as static collections often benefit from modeling topics, dynamic collections will likely benefit from temporal modeling of events and time-sensitive user interests and intents, which were rarely addressed in the literature. There have been preliminary efforts in the research and industrial communities to address algorithms, architectures, evaluation methodologies and metrics. We aim to bring together practitioners and researchers to discuss their recent breakthroughs and the challenges with addressing time-aware information access, both from the algorithmic and the architectural perspectives. This workshop is a successor to the successful SIGIR 2012 Workshop on Time Aware Information Access (#TAIA2012). Where the 2012 edition was the first to bring together a broad set of academic and industrial researchers around the topic of time-aware information access, the specific focus of this workshop is on the many time-aware benchmarking activities that are ongoing in 2013.
Live nuggets extractor: a semi-automated system for text extraction and test collection creation
Authors: Matthew Ekstrand-Abueg, Virgil Pavlu, Javed A. Aslam
Venue: SIGIR
Year: 2013

The Live Nugget Extractor system provides users with a method of efficiently and accurately collecting relevant information for any web query rather than providing a simple ranked lists of documents. The system utilizes an online learning procedure to infer relevance of unjudged documents while extracting and ranking information from judged documents. This creates a set of judged and inferred relevance scores for both documents and text fragments, which can be used for test collections, summarization, and other tasks where high accuracy and large collections with minimal human effort are needed.
Exploring semi-automatic nugget extraction for Japanese one click access evaluation
Authors: Matthew Ekstrand-Abueg, Virgil Pavlu, Makoto P. Kato, Tetsuya Sakai, Takehiro Yamamoto, Mayu Iwata
Venue: SIGIR
Year: 2013

Building test collections based on nuggets is useful evaluating systems that return documents, answers, or summaries. However, nugget construction requires a lot of manual work and is not feasible for large query sets. Towards an efficient and scalable nugget-based evaluation, we study the applicability of semi-automatic nugget extraction in the context of the ongoing NTCIR One Click Access (1CLICK) task. We compare manually-extracted and semi-automatically-extracted Japanese nuggets to demonstrate the coverage and efficiency of the semi-automatic nugget extraction. Our findings suggest that the manual nugget extraction can be replaced with a direct adaptation of the English semi-automatic nugget extraction system, especially for queries for which the user desires broad answers from free-form text.
X-ENS: semantic enrichment of web search results at real-time
Authors: Pavlos Fafalios, Yannis Tzitzikas
Venue: SIGIR
Year: 2013

While more and more semantic data are published on the Web, an important question is how typical web users can access and exploit this body of knowledge. Although, existing interaction paradigms in semantic search hide the complexity behind an easy-to-use interface, they have not managed to cover common search needs. In this paper, we present X-ENS (eXplore ENtities in Search), a web search application that enhances the classical, keyword-based, web searching with semantic information, as a means to combine the pros of both Semantic Web standards and common Web Searching. X-ENS identifies entities of interest in the snippets of the top search results which can be further exploited in a faceted interaction scheme, and thereby can help the user to limit the - often very large - search space to those hits that contain a particular piece of information. Moreover, X-ENS permits the exploration of the identified entities by exploiting semantic repositories.
Collaborative factorization for recommender systems
Authors: Chaosheng Fan, Yanyan Lan, Jiafeng Guo, Zuoquan Lin, Xueqi Cheng
Venue: SIGIR
Year: 2013

Recommender system has become an effective tool for information filtering, which usually provides the most useful items to users by a top-k ranking list. Traditional recommendation techniques such as Nearest Neighbors (NN) and Matrix Factorization (MF) have been widely used in real recommender systems. However, neither approaches can well accomplish recommendation task since that: (1) most NN methods leverage the neighbor's behaviors for prediction, which may suffer the severe data sparsity problem; (2) MF methods are less sensitive to sparsity, but neighbors' influences on latent factors are not fully explored, since the latent factors are often used independently. To overcome the above problems, we propose a new framework for recommender systems, called collaborative factorization. It expresses the user as the combination of his own factors and those of the neighbors', called collaborative latent factors, and a ranking loss is then utilized for optimization. The advantage of our approach is that it can both enjoy the merits of NN and MF methods. In this paper, we take the logistic loss in RankNet and the likelihood loss in ListMLE as examples, and the corresponding collaborative factorization methods are called CoF-Net and CoF-MLE. Our experimental results on three benchmark datasets show that they are more effective than several state-of-the-art recommendation methods.
Answering natural language queries over linked data graphs: a distributional semantics approach
Authors: André Freitas, Fabrício Firmino de Faria, Seán O&apos, Riain, Edward Curry
Venue: SIGIR
Year: 2013

This paper demonstrates Treo, a natural language query mechanism for Linked Data graphs. The approach uses a distributional semantic vector space model to semantically match user query terms with data, supporting vocabulary-independent (or schema-agnostic) queries over structured data.
Segmentation strategies for passage retrieval in audio-visual documents
Authors: Petra Galuscáková
Venue: SIGIR
Year: 2013

The importance of Information Retrieval (IR) in audio-visual recordings has been increasing with steeply growing numbers of audio-visual documents available on-line. Compared to traditional IR methods, this task requires specific techniques, such as Passage Retrieval which can accelerate the search process by retrieving the exact relevant passage of a recording instead of the full document. In Passage Retrieval, full recordings are divided into shorter segments which serve as individual documents for the further IR setup. This technique also allows normalizing document length and applying positional information. It was shown that it can even improve retrieval results.
An LDA-smoothed relevance model for document expansion: a case study for spoken document retrieval
Authors: Debasis Ganguly, Johannes Leveling, Gareth J. F. Jones
Venue: SIGIR
Year: 2013

Document expansion (DE) in information retrieval (IR) involves modifying each document in the collection by introducing additional terms into the document. It is particularly useful to improve retrieval of short and noisy documents where the additional terms can improve the description of the document content. Existing approaches to DE assume that documents to be expanded are from a single topic. In the case of multi-topic documents this can lead to a topic bias in terms selected for DE and hence may result in poor retrieval quality due to the lack of coverage of the original document topics in the expanded document. This paper proposes a new DE technique providing a more uniform selection and weighting of DE terms from all constituent topics. We show that our proposed method significantly outperforms the most recently reported relevance model based DE method on a spoken document retrieval task for both manual and automatic speech recognition transcripts.
Mapping queries to questions: towards understanding users' information needs
Authors: Yunjun Gao, Lu Chen, Rui Li, Gang Chen
Venue: SIGIR
Year: 2013

In this paper, for the first time, we study the problem of mapping keyword queries to questions on community-based question answering (CQA) sites. Mapping general web queries to questions enables search engines not only to discover explicit and specific information needs (questions) behind keywords queries, but also to find high quality information (answers) for answering keyword queries. In order to map queries to questions, we propose a ranking algorithm containing three steps: Candidate Question Selection, Candidate Question Ranking, and Candidate Question Grouping. Preliminary experimental results using 60 queries from search logs of a commercial engine show that the presented approach can efficiently find the questions which capture user's information needs explicitly.
Internet advertising: theory and practice
Authors: Bin Gao, Jun Yan, Dou Shen, Tie-Yan Liu
Venue: SIGIR
Year: 2013

Internet advertising, a form of advertising that utilizes the Internet to deliver marketing messages and attract customers, has seen exponential growth since its inception around twenty years ago; it has been pivotal to the success of the World Wide Web. The dramatic growth of internet advertising poses great challenges to information retrieval, machine learning, data mining and game theory, and it calls for novel technologies to be developed. The main purpose of this workshop is to bring together researchers and practitioners in the area of Internet Advertising and enable them to share their latest research results, to express their opinions, and to discuss future directions.
A mutual information-based framework for the analysis of information retrieval systems
Authors: Peter B. Golbus, Javed A. Aslam
Venue: SIGIR
Year: 2013

We consider the problem of information retrieval evaluation and the methods and metrics used for such evaluations. We propose a probabilistic framework for evaluation which we use to develop new information-theoretic evaluation metrics. We demonstrate that these new metrics are powerful and generalizable, enabling evaluations heretofore not possible.
From keywords to keyqueries: content descriptors for the web
Authors: Tim Gollub, Matthias Hagen, Maximilian Michel, Benno Stein
Venue: SIGIR
Year: 2013

We introduce the concept of keyqueries as dynamic content descriptors for documents. Keyqueries are defined implicitly by the index and the retrieval model of a reference search engine: keyqueries for a document are the minimal queries that return the document in the top result ranks. Besides applications in the fields of information retrieval and data mining, keyqueries have the potential to form the basis of a dynamic classification system for future digital libraries---the modern version of keywords for content description.