OpenRaiser/Intern-Atlas huggingface dataset:
Efficient query recommendations in the long tail via center-piece subgraphs
Authors: Francesco Bonchi, Raffaele Perego, Fabrizio Silvestri, Hossein Vahabi, Rossano Venturini
Venue: SIGIR
Year: 2012

We present a recommendation method based on the well-known concept of center-piece subgraph, that allows for the time/space efficient generation of suggestions also for rare, i.e., long-tail queries. Our method is scalable with respect to both the size of datasets from which the model is computed and the heavy workloads that current web search engines have to deal with. Basically, we relate terms contained into queries with highly correlated queries in a query-flow graph. This enables a novel recommendation generation method able to produce recommendations for approximately 99% of the workload of a real-world search engine. The method is based on a graph having term nodes, query nodes, and two kinds of connections: term-query and query-query. The first connects a term to the queries in which it is contained, the second connects two query nodes if the likelihood that a user submits the second query after having issued the first one is sufficiently high. On such large graph we need to compute the center-piece subgraph induced by terms contained into queries. In order to reduce the cost of the above computation, we introduce a novel and efficient method based on an inverted index representation of the model. We experiment our solution on two real-world query logs and we show that its effectiveness is comparable (and in some case better) than state-of-the-art methods for head-queries. More importantly, the quality of the recommendations generated remains very high also for long-tail queries, where other methods fail even to produce any suggestion. Finally, we extensively investigate scalability and efficiency issues and we show the viability of our method in real world search engines.
A knowledge-based approach for summarising opinions
Authors: Marco Bonzanini
Venue: SIGIR
Year: 2012

Automatic Document Summarisation plays a central role in the process of providing the user with a quick access to information. Applications range from the generation of news headlines, to the aggregation of opinions extracted from reviews. Traditional topic-based summarisation systems are not always able to capture the sentiments expressed in a review. Major efforts in sentiment analysis have been put in the tasks of mining and classifying reviews according to their polarity. In this research, we investigate the use of summarisation techniques applied to reviews, and we propose a knowledge-based approach to summarisation, in the context of sentiment analysis. The proposed research is focused on three different aspects. Firstly, we investigate the application of summarisation techniques to sentiment classification. Capturing the key passage of a review can be beneficial for both a sentiment classifier, and for a user who could potentially understand the polarity of a review without reading the full text. Secondly, we investigate how to combine knowledge extracted from the reviews or integrated from external sources, with the purpose of producing opinion-oriented summaries. Thirdly, we analyse the possibility of generating personalised (user-oriented or query-biased) opinion-based summaries.
Opinion summarisation through sentence extraction: an investigation with movie reviews
Authors: Marco Bonzanini, Miguel Martinez-Alvarez, Thomas Roelleke
Venue: SIGIR
Year: 2012

In on-line reviews, authors often use a short passage to describe the overall feeling about a product or a service. A review as a whole can mention many details not in line with the overall feeling, so capturing this key passage is important to understand the overall sentiment of the review. This paper investigates the use of extractive summarisation in the context of sentiment classification. The aim is to find the summary sentence, or the short passage, which gives the overall sentiment of the review, filtering out potential noisy information. Experiments on a movie review data-set show that subjectivity detection plays a central role in building summaries for sentiment classification. Subjective extracts carry the same polarity of the full text reviews, while statistical and positional approaches are not able to capture this aspect.
IR paradigms in computational advertising
Authors: Andrei Z. Broder
Venue: SIGIR
Year: 2012

The central problem in the emerging discipline of computational advertising is to find the "best match" between a given user in a given context and a suitable advertisement. The context could be a user entering a query in a search engine ("sponsored search"), a user reading a web page ("content match" and "display ads"), a user streaming a movie, and so on. In some situations, it is desirable to solve the "dual" optimization problem: rather then find the best ad given a user in a context, the goal is to identify the "best audience", i.e. the most receptive set of users and/or the most suitable contexts for a given advertising campaign. The information about the user can vary from scarily detailed to practically nil. The number of potential advertisements might be in the billions. Thus, depending on the definition of "best match" and "best audience" these problems lead to a variety of massive optimization problems, with complicated constraints, and challenging data representation and access issues.
A subjunctive exploratory search interface to support media studies researchers
Authors: Marc Bron, Jasmijn van Gorp, Frank Nack, Maarten de Rijke, Andrei Vishneuski, Sonja de Leeuw
Venue: SIGIR
Year: 2012

Media studies concerns the study of production, content, and/or reception of various types of media. Today's continuous production and storage of media is changing the way media studies researchers work and requires the development of new search models and tools.
Watson: the Jeopardy! challenge and beyond
Authors: Eric W. Brown
Venue: SIGIR
Year: 2012

Watson, named after IBM founder Thomas J. Watson, was built by a team of IBM researchers who set out to accomplish a grand challenge-build a computing system that rivals a human's ability to answer questions posed in natural language with speed, accuracy and confidence. The quiz show Jeopardy! provided the ultimate test of this technology because the game's clues involve analyzing subtle meaning, irony, riddles and other complexities of natural language in which humans excel and computers traditionally fail. Watson passed its first test on Jeopardy!, beating the show's two greatest champions in a televised exhibition match, but the real test will be in applying the underlying natural language processing and analytics technology in business and across industries. In this talk I will introduce the Jeopardy! grand challenge, present an overview of Watson and the DeepQA technology upon which Watson is built, and explore future applications of this technology.
Incorporating statistical topic information in relevance feedback
Authors: Karla L. Caballero Espinosa, Ram Akella
Venue: SIGIR
Year: 2012

Most of the relevance feedback algorithms only use document terms as feedback (local features) in order to update the query and re-rank the documents to show to the user. This approach is limited by the terms of those documents without any global context. We propose to use statistical topic modeling techniques in relevance feedback to incorporate a better estimate of context by including global information about the document. This is particularly helpful for difficult queries where learning the context from the interactions with the user is crucial. We propose to use the topic mixture information obtained to characterize the documents and learn their topics. Then, we rank documents incorporating positive and negative feedback by fitting a latent distribution for each class of documents online and combining all the features using Bayesian Logistic Regression. We show results using the OHSUMED dataset for 3 different variants and obtain higher performance, up to 12.5% in Mean Average Precision (MAP).
Query performance prediction for IR
Authors: David Carmel, Oren Kurland
Venue: SIGIR
Year: 2012

The goal of this tutorial is to expose participants to current research on query performance prediction. Participants will become familiar with state-of-the-art performance prediction methods, with common evaluation methodologies of prediction quality, and with potential applications that can utilize performance predictors. In addition, some open issues and challenges in the field will be discussed.
Advances on the development of evaluation measures
Authors: Ben Carterette, Evangelos Kanoulas, Emine Yilmaz
Venue: SIGIR
Year: 2012

The goal of the tutorial is to provide attendees with a comprehensive overview of the latest advances in the development of information retrieval evaluation measures and discuss the current challenges in the area. A number of topics are covered, including background in traditional evaluation paradigm and traditional evaluation measures, evaluation measures based on user models, advanced models of user interaction with search engines, measures based on these models, measures for novelty and diversity, and session-based measures.
A framework for manipulating and searching multiple retrieval types
Authors: Marc-Allen Cartright, Ethem F. Can, William Dabney, Jeff Dalton, Logan Giorda, Kriste Krstovski, Xiaoye Wu, Ismet Zeki Yalniz, James Allan, R. Manmatha, David A. Smith
Venue: SIGIR
Year: 2012

Conventional retrieval systems view documents as a unit and look at different retrieval types within a document. We introduce Proteus, a frame-work for seamlessly navigating books as dynamic collections which are defined on the fly. Proteus allows us to search various retrieval types. Navigable types include pages, books, named persons, locations, and pictures in a collection of books taken from the Internet Archive. The demonstration shows the value of multi-type browsing in dynamic collections to peruse new data.
Distilling and exploring nuggets from a corpus
Authors: Vittorio Castelli, Hema Raghavan, Radu Florian, Ding-Jung Han, Xiaoqiang Luo, Salim Roukos
Venue: SIGIR
Year: 2012

This paper describes a live and scalable system that automatically extracts information nuggets for entities/topics from a continuously updated corpus for effective exploration and analysis. A nugget is a piece of semantic information that (1) must be mapped semantically to the transitive closure of a pre-defined ontology, (2) is explicitly supported by text, and (3) has a natural language description that completely conveys its semantic to a user. Fig. 1 shows a type of nugget "involvement in events" for a person entity (Leon Panetta): each nugget has a short description ("meeting", "news conference") with a list of supporting passages. Our key contributions are (1) We extract nuggets and remove redundancy to produce a summary of salient information with supporting clusters of passages. (2) We present an entity/topic centric exploration interface that also allows users to navigate to other entities involved in a nugget. (3) We use the statistical NLP technologies developed over the years in the ACE ,GALE and TAC-KBP programs, including parsing, mention detection, within and cross document coreference resolution, relation detection and slot filler extraction. (4) Our system is flexible and easily adaptable across domains as demonstrated on two corpora: generic news and scientific papers. Search engines such as Google News and Scholar do not retrieve nuggets, and only remove redundancy at document level. News aggregation applications such as Evri categorize news articles based on the entities of topics but do not extract nuggets. Other systems extract richer information, but not all of it has clear semantics; e.g., Silobreaker presents results as "the relationship between X and Y in the context of [keyphrase]", leaving users with the task of interpreting the semantics as it is not tied to a clear ontology. In contrast we remove redundancy, summarize results and present nuggets that have clear semantics.
Social-network analysis using topic models
Authors: Youngchul Cha, Junghoo Cho
Venue: SIGIR
Year: 2012

In this paper, we discuss how we can extend probabilistic topic models to analyze the relationship graph of popular social-network data, so that we can group or label the edges and nodes in the graph based on their topic similarity. In particular, we first apply the well-known Latent Dirichlet Allocation (LDA) model and its existing variants to the graph-labeling task and argue that the existing models do not handle popular nodes (nodes with many incoming edges) in the graph very well. We then propose possible extensions to this model to deal with popular nodes. Our experiments show that the proposed extensions are very effective in labeling popular nodes, showing significant improvements over the existing methods. Our proposed methods can be used for providing, for instance, more relevant friend recommendations within a social network.
Using preference judgments for novel document retrieval
Authors: Praveen Chandar, Ben Carterette
Venue: SIGIR
Year: 2012

There has been considerable interest in incorporating diversity in search results to account for redundancy and the space of possible user needs. Most work on this problem is based on subtopics: diversity rankers score documents against a set of hypothesized subtopics, and diversity rankings are evaluated by assigning a value to each ranked document based on the number of novel (and redundant) subtopics it is relevant to. This can be seen as modeling a user who is always interested in seeing more novel subtopics, with progressively decreasing interest in seeing the same subtopic multiple times. We put this model to test: if it is correct, then users, when given a choice, should prefer to see a document that has more value to the evaluation. We formulate some specific hypotheses from this model and test them with actual users in a novel preference-based design in which users express a preference for document A or document B given document C. We argue that while the user study shows the subtopic model is good, there are many other factors apart from novelty and redundancy that may be influencing user preferences. From this, we introduce a new framework to construct an ideal diversity ranking using only preference judgments, with no explicit subtopic judgments whatsoever.
Using PageRank to infer user preferences
Authors: Praveen Chandar, Ben Carterette
Venue: SIGIR
Year: 2012

Recently, researchers have shown interest in the use of preference judgments for evaluation in IR literature. Although preference judgments have several advantages over absolute judgment, one of the major disadvantages is that the number of judgments needed increases polynomially as the number of documents in the pool increases. We propose a novel method using PageRank to minimize the number of judgments required to evaluate systems using preference judgments. We test the proposed hypotheses using the TREC 2004 to 2006 Terabyte dataset to show that it is possible to reduce the evaluation cost considerably. Further, we study the susceptibility of the methods due to assessor errors.
Learning-based time-sensitive re-ranking for web search
Authors: Po-Tzu Chang, Yen-Chieh Huang, Cheng-Lun Yang, Shou-De Lin, Pu-Jen Cheng
Venue: SIGIR
Year: 2012

To model time-dependent user intent for Web search, this paper proposes a novel method using machine learning techniques to exploit temporal features for effective time-sensitive search result re-ranking. We propose models to incorporate users' click through information for queries that are seen in the training data, and then further extend the model to deal with unseen queries considering the relationship between queries. Experiment shows significant improvement on search result ranking over original search outputs.
Finding web appearances of social network users via latent factor model
Authors: Kailong Chen, Zhengdong Lu, Xiaoshi Yin, Yong Yu, Zaiqing Nie
Venue: SIGIR
Year: 2012

With the rapid growing of Web 2.0, people spend more time on social networks such as Facebook and Twitter. In order to know the people they are interacting with, finding the web appearances of them will help the social network users to a great extent. We propose a novel and effective latent factor model to find web appearances of target social network users. Our method solves the name ambiguity problem by simultaneously exploring the link structure of social networks and the web. Experiments on real-world data show the superiority of our method over several baselines.
Enhancing knowledge base with knowledge transfer
Authors: Si-Chi Chin
Venue: SIGIR
Year: 2012

A Knowledge Base (KB) stores, organizes, and shares information pertinent to entities (i.e. KB nodes) such as people, organizations, and events. A large KB system, such as Wikipedia, relies on human curators to create and maintain the content in the systems. However, it has become challenging for human curators to sift through the rapidly growing amount of information and filter out the information irrelevant to a KB node. The area of Knowledge Base Enhancement (KBE) aims to explore and identify automatic methods to assist human curators to accelerate the process. KBE can be viewed as a special case of Information Filtering (IF). However, the lack of high-quality labelled data introduces a major challenge to train a satisfying model for the task. Transfer learning provides solutions to the problem and has explored applications in the area of text mining, whereas direct application to KBE or IF remains absent.
Survival analysis of click logs
Authors: Si-Chi Chin, W. Nick Street
Venue: SIGIR
Year: 2012

Click logs from search engines provide a rich opportunity to acquire implicit feedback from users. Patterns derived from the time between a posted query and a click provide information on the ranking quality, reflecting the perceived relevance of a retrieved URL. This paper applies the Kaplan-Meier estimator to study click patterns. The visualization of click curves demonstrates the interaction between the relevance and the rank position of URLs. The observed results demonstrate the potential of using click curves to predict the quality of the top-ranked results.
Dependency trigram model for social relation extraction from news articles
Authors: Maengsik Choi, Harksoo Kim, W. Bruce Croft
Venue: SIGIR
Year: 2012

We propose a kernel-based model to automatically extract social relations such as economic relations and political relations between two people from news articles. To determine whether two people are structurally associated with each other, the proposed model uses an SVM (support vector machine) tree kernel based on trigrams of head-dependent relations between them. In the experiments with the automatic content extraction (ACE) corpus and a Korean news corpus, the proposed model outperformed the previous systems based on SVM tree kernels even though it used more shallow linguistic knowledge.
How query extensions reflect search result abandonments
Authors: Aleksandr Chuklin, Pavel Serdyukov
Venue: SIGIR
Year: 2012

It is often considered that high abandonment rate corresponds to poor IR system performance. However several studies suggested that there are so called good abandonments, i.e. situations when search engine result page contains enough details to satisfy the user information need without necessity to click on search results. In this work we propose to look at query extensions. We see that an extension by itself might motivate abandonment type (good or bad) for the underlying query to some degree. We also propose a way to find potentially good abandonment extensions in an automated manner.
Explaining neighborhood-based recommendations
Authors: Sergio Cleger-Tamayo, Juan M. Fernández-Luna, Juan F. Huete
Venue: SIGIR
Year: 2012

Recommender Systems (RS) attempt to discover users' preferences, and to learn about them in order to anticipate their needs. The main task normally associated with a RS is to offer suggestions for items. However, for most users, RSs are black boxes, computerized oracles that give advice, but cannot be questioned. In order to improve the quality of predictions and the satisfaction of the users, explanations facilities are needed. We present a novel methodology to explain recommendations: showing predictions over a set of observed items. Our proposal has been validated by means of user studies and lab experiments using MovieLens dataset.
Efficient in-memory top-k document retrieval
Authors: J. Shane Culpepper, Matthias Petri, Falk Scholer
Venue: SIGIR
Year: 2012

For over forty years the dominant data structure for ranked document retrieval has been the inverted index. Inverted indexes are effective for a variety of document retrieval tasks, and particularly efficient for large data collection scenarios that require disk access and storage. However, many efficiency-bound search tasks can now easily be supported entirely in memory as a result of recent hardware advances. In this paper we present a hybrid algorithmic framework for in-memory bag of-words ranked document retrieval using a self-index derived from the FM-Index, wavelet tree, and the compressed suffix tree data structures, and evaluate the various algorithmic trade-offs for performing efficient queries entirely in-memory. We compare our approach with two classic approaches to bag-of-words queries using inverted indexes, term-at-a-time (TAAT) and document-at-a-time (DAAT) query processing. We show that our framework is competitive with state-of-the-art indexing structures, and describe new capabilities provided by our algorithms that can be leveraged by future systems to improve effectiveness and efficiency for a variety of fundamental search operations.
Investigating performance predictors using monte carlo simulation and score distribution models
Authors: Ronan Cummins
Venue: SIGIR
Year: 2012

The standard deviation of scores in the top k documents of a ranked list has been shown to be significantly correlated with average precision and has been the basis of a number of query performance predictors. In this paper, we outline two hypotheses that aid in understanding this correlation. Using score distribution (SD) models with known parameters, we create a large number of document rankings using Monte Carlo simulation to test the validity of these hypotheses.
Putting context into search and search into context
Authors: Susan T. Dumais
Venue: SIGIR
Year: 2012

It is very challenging task to understand a short query, especially if that query is considered in isolation. Luckily, queries do magically appear in a search box -- rather, they are issued by real people, trying to accomplish a task, at a given point in time and space, and this "context" can be used to aid query understanding. Traditionally search engines have returned the same results to everyone who asks the same question. However, using a single ranking for everyone, in every context limits how well a search engine can do. In this talk I outline a framework to quantify the "potential for personalization", that can be used to characterize the extent to which different people have the same (or different) intents for a query. I then describe several examples of how we represent and use different kinds of context to improve search quality. Finally I conclude by highlighting some important challenges in developing such systems at Web scale including system optimization, evaluation, transparency and serendipity.