OpenRaiser/Intern-Atlas huggingface dataset:
User Behavior in Asynchronous Slow Search
Authors: Ryan Burton, Kevyn Collins-Thompson
Venue: SIGIR
Year: 2016

Conventional Web search is predicated on returning results to users as quickly as possible. However, for some search tasks, users have reported a willingness to wait for the perfect set of results. In this work, we present the first study to analyze users' willingness to wait and their search success, when given a Web search system that embodies characteristics of slow search, where speed can be traded for an improvement in quality. We conducted a between-subjects user study involving tasks that required multiple queries to complete, providing a Web search system that gave users the option to additionally issue asynchronous queries for which results improve in relevance over time as users continued working. We analyze the resulting survey results and interaction log data to investigate how users spent their time while waiting, and how behavior and search outcomes changes when users are given the option of using a system with asynchronous slow search capabilities. We find that when given a slow search system, users are able to perceive the improvement in quality over time, and find tasks to be easier compared to a baseline conventional Web search system. Additionally, we find that users continue to issue their own queries and examine additional documents while the slow search queries are processed in the background, and use the slow search feature more effectively as they gain exposure to its behavior across tasks. Our study significantly advances our understanding of the benefits and tradeoffs involved in providing slow search scenarios for Web search.
Scalability and Efficiency Challenges in Large-Scale Web Search Engines
Authors: Berkant Barla Cambazoglu, Ricardo Baeza-Yates
Venue: SIGIR
Year: 2016

Commercial web search engines need to process thousands of queries every second and provide responses to user queries within a few hundred milliseconds. As a consequence of these tight performance constraints, search engines construct and maintain very large computing infrastructures for crawling the Web, indexing discovered pages, and processing user queries. The scalability and efficiency of these infrastructures require careful performance optimizations in every major component of the search engine.
A Complete & Comprehensive Movie Review Dataset (CCMR)
Authors: Xuezhi Cao, Weiyue Huang, Yong Yu
Venue: SIGIR
Year: 2016

Online review sites are widely used for various domains including movies and restaurants. These sites now have strong influences towards users during purchasing processes. There exist plenty of research works for review sites on various aspects, including item recommendation, user behavior analysis, etc. However, due to the lack of complete and comprehensive dataset, there are still problems that remain to be solved. Therefore, in this paper we assemble and publish such dataset (CCMR) for the community. CCMR outruns existing datasets in terms of completeness, comprehensiveness and scale. Besides describing the dataset and its collecting methodology, we also propose several potential research topics that are made possible by having this dataset. Such topics include: (i) a statistical approach to reduce the impacts from fake reviews and (ii) analyzing and modeling the influences of public opinions towards users during rating actions. We further conduct preliminary analysis and experiments for both directions to show that they are promising.
Evaluating Retrieval over Sessions: The TREC Session Track 2011-2014
Authors: Ben Carterette, Paul D. Clough, Mark M. Hall, Evangelos Kanoulas, Mark Sanderson
Venue: SIGIR
Year: 2016

Information Retrieval (IR) research has traditionally focused on serving the best results for a single query - so-called ad hoc retrieval. However, users typically search iteratively, refining and reformulating their queries during a session. A key challenge in the study of this interaction is the creation of suitable evaluation resources to assess the effectiveness of IR systems over sessions. This paper describes the TREC Session Track, which ran from 2010 through to 2014, which focussed on forming test collections that included various forms of implicit feedback. We describe the test collections; a brief analysis of the differences between datasets over the years; and the evaluation results that demonstrate that the use of user session data significantly improved effectiveness.
Ranking Financial Tweets
Authors: Diego Ceccarelli, Francesco Nidito, Miles Osborne
Venue: SIGIR
Year: 2016

Recently Twitter has complemented traditional newswire as a source of valuable Financial information. Although there is a rich body of published research dealing with the task of ranking tweets, there has been little published research dealing with ranking tweets within a Financial context. Here we consider whether popularity factors within Twitter can be used as a signal for popularity within the domain of financial experts. Our results suggest that what interests Finance is not the same as what interests the users of Twitter.
Where the Event Lies: Predicting Event Occurrence in Textual Documents
Authors: Andrea Ceroni, Ujwal Gadiraju, Jan Matschke, Simon Wingert, Marco Fisichella
Venue: SIGIR
Year: 2016

Manually inspecting text in a document collection to assess whether an event occurs in it is a cumbersome task. Although a manual inspection can allow one to identify and discard false events, it becomes infeasible with increasing numbers of automatically detected events. In this paper, we present a system to automatize event validation, defined as the task of determining whether a given event occurs in a given document or corpus. In addition to supporting users seeking for information that corroborates a given event, event validation can also boost the precision of automatically detected event sets by discarding false events and preserving the true ones. The system allows to specify events, retrieves candidate web documents, and assesses whether events occur in them. The validation results are shown to the user, who can revise the decision of the system. The validation method relies on a supervised model to predict the occurrence of events in a non-annotated corpus. This system can also be used to build ground-truths for event corpora.
Learning to Rank Features for Recommendation over Multiple Categories
Authors: Xu Chen, Zheng Qin, Yongfeng Zhang, Tao Xu
Venue: SIGIR
Year: 2016

Incorporating phrase-level sentiment analysis on users' textual reviews for recommendation has became a popular meth-od due to its explainable property for latent features and high prediction accuracy. However, the inherent limitations of the existing model make it difficult to (1) effectively distinguish the features that are most interesting to users, (2) maintain the recommendation performance especially when the set of items is scaled up to multiple categories, and (3) model users' implicit feedbacks on the product features. In this paper, motivated by these shortcomings, we first introduce a tensor matrix factorization algorithm to Learn to Rank user Preferences based on Phrase-level sentiment analysis across Multiple categories (LRPPM for short), and then by combining this technique with Collaborative Filtering (CF) method, we propose a novel model called LRPPM-CF to boost the performance of recommendation. Thorough experiments on two real-world datasets demonstrate that our proposed model is able to improve the performance in the tasks of capturing users' interested features and item recommendation by about 17%-24% and 7%-13%, respectively, as compared with several state-of-the-art methods.
An Empirical Study of Learning to Rank for Entity Search
Authors: Jing Chen, Chenyan Xiong, Jamie Callan
Venue: SIGIR
Year: 2016

This work investigates the effectiveness of learning to rank methods for entity search. Entities are represented by multi-field documents constructed from their RDF triples, and field-based text similarity features are extracted for query-entity pairs. State-of-the-art learning to rank methods learn models for ad-hoc entity search. Our experiments on an entity search test collection based on DBpedia confirm that learning to rank methods are as powerful for ranking entities as for ranking documents, and establish a new state-of-the-art for accuracy on this benchmark dataset.
Linking Organizational Social Network Profiles
Authors: Jerome Cheng, Kazunari Sugiyama, Min-Yen Kan
Venue: SIGIR
Year: 2016

Many organizations possess social media accounts on different social networks, but these profiles are not always linked. End applications, users, as well as the organization themselves, can benefit when the profiles are appropriately identified and linked. Most existing works on social network entity linking focus on linking individuals, and do not model features specific for organizational linking. We address this gap not only to link official social media accounts but also to discover and solve the identification and linking of associated affiliate accounts -- such as geographical divisions and brands -- which are important to distinguish.
A Study of Information Seeking Behavior Using Physical and Online Explorations
Authors: Dongho Choi
Venue: SIGIR
Year: 2016

People have their behavioral patterns, through which they determine how to seek and use information. People also exhibit established mobility pattern in their everyday lives. Meanwhile, the modern technologies such as smartphones, wearable devices, and eye trackers have allowed researchers to collect personal, contextual, and cognitive information of users, and create behavioral models from different perspectives. Considering the analogy between information exploration and geographical exploration, I want to identify the interconnections between these behaviors and predict individuals? search behavior using personal and contextual signals. The proposal uses a mixed-method approach that involves a four-week field study, a game study, and a lab study, collecting data from 40 participants through mobile device, eye tracker, online logs, and weekly diary.
Appearance-Based Retrieval of Mathematical Notation in Documents and Lecture Videos
Authors: Kenny Davila
Venue: SIGIR
Year: 2016

Large data collections containing millions of math formulae in different formats are available on-line. Retrieving math expressions from these collections is challenging. Based on the notion that visually similar formulas are related, we propose a framework for appearance-based formula retrieval in two different modalities: symbolic for text documents and image-Based for videos.
Search Result Prefetching Using Cursor Movement
Authors: Fernando Diaz, Qi Guo, Ryen W. White
Venue: SIGIR
Year: 2016

Search result examination is an important part of searching. High page load latency for landing pages (clicked results) can reduce the efficiency of the search process. Proactively prefetching landing pages in advance of clickthrough can save searchers valuable time. However, prefetching consumes resources that are wasted unless the prefetched results are requested by searchers. Balancing the costs in prefetching particular results against the benefits in reduced latency to searchers represents the search result prefetching challenge. We present methods that leverage searchers' cursor movements on search result pages in real time to dynamically estimate the result that searchers will request next. We demonstrate through large-scale log analysis that our approach significantly outperforms three strong baselines that prefetch results based on (i) the search engine result ranking, (ii) past clicks from all searchers for the query, or (iii) past clicks from the current searcher for the query. Our promising findings have implications for the design of search support that makes the search process more efficient.
"Is Sven Seven?": A Search Intent Module for Children
Authors: Nevena Dragovic, Ion Madrazo Azpiazu, Maria Soledad Pera
Venue: SIGIR
Year: 2016

The Internet is the biggest data-sharing platform, comprised of an immeasurable quantity of resources covering diverse topics appealing to users of all ages. Children shape tomorrow's society, so it is essential that this audience becomes agile with searching information. Although young users prefer well-known search engines, their lack of skill in formulating adequate queries and the fact that search tools were not designed explicitly with children in mind, can result in poor outcomes. The reasons for this include children's limited vocabulary, which makes it challenging to articulate information needs using short queries, or their tendency to create queries that are too long, which translates to few or irrelevant retrieved results. To enhance web search environments in response to children's behaviors and expectations, in this paper we discuss an initial effort to verify well-known issues, and identify yet to be explored ones, that affect children in formulating (natural language or keyword) queries. We also present a novel search intent module developed in response to these issues, which can seamlessly be integrated with existing search engines favored by children. The proposed module interprets a child's query and creates a shorter and more concise query to submit to a search engine, which can lead to a more successful search session. Initial experiments conducted using a sample of children queries validate the correctness of the proposed search intent module.
DAJEE: A Dataset of Joint Educational Entities for Information Retrieval in Technology Enhanced Learning
Authors: Vladimir Estivill-Castro, Carla Limongelli, Matteo Lombardi, Alessandro Marani
Venue: SIGIR
Year: 2016

In the Technology Enhanced Learning (TEL) community, the problem of conducting reproducible evaluations of recommender systems is still open, due to the lack of exhaustive benchmarks. The few public datasets available in TEL have limitations, being mostly small and local.
Automatic Identification and Contextual Reformulation of Implicit System-Related Queries
Authors: Adam Fourney, Susan T. Dumais
Venue: SIGIR
Year: 2016

Web search functionality is increasingly integrated into operating systems, software applications, and other interactive environments that extend beyond the traditional web browser. In particular, intelligent virtual assistants (e.g., Microsoft Cortana or Apple Siri) often "fall-back" to generic web search in cases where utterances fall outside the set of scenarios known to the agent. In this paper we analyze a 3 month log of web search queries posed via the Cortana virtual assistant. We report that, in this environment, users frequently ask questions that implicitly pertain to the systems or devices from which they are searching (e.g., asking: [how do I take a screenshot]). Unfortunately, accurately answering these implicit system queries poses significant challenges to general web search engines, due in part to the lack of available context. We show that such queries: (1) can be detected with high precision, (2) are common, and (3) can be automatically reformulated to substantially improve retrieval performance in these fall-through scenarios.
Constructing and Mining Web-scale Knowledge Graphs
Authors: Evgeniy Gabrilovich, Nicolas Usunier
Venue: SIGIR
Year: 2016

Recent years have witnessed a proliferation of large-scale knowledge graphs, from purely academic projects such as YAGO to major commercial projects such as Google's Knowledge Graph and Microsoft's Satori. Whereas there is a large body of research on mining homogeneous graphs, this new generation of information networks are highly heterogeneous, with thousands of entity and relation types and billions of instances of those types (graph vertices and edges). In this tutorial, we present the state of the art in constructing, mining, and growing knowledge graphs. The purpose of the tutorial is to equip newcomers to this exciting field with an understanding of the basic concepts, tools and methodologies, open research challenges, as well as pointers to available datasets and relevant literature. Knowledge graphs have become an enabling resource for a plethora of new knowledge-rich applications. Consequently, the tutorial will also discuss the role of knowledge bases in empowering a range of web applications, from web search to social networks to digital assistants. A publicly available knowledge base (Freebase) will be used throughout the tutorial to exemplify the different techniques.
Torii: Attribute-based Polarity Analysis with Big Datasets
Authors: Fernando O. Gallego
Venue: SIGIR
Year: 2016

Polarity analysis has become a key aspect of market analysis. The number of companies that are interested in the general opinion of the crowd regarding the items that they sell is increasing everyday. Attribute-based polarity analysis is a fine-grained approach that computes if the opinion about an attribute of (a component of) an item is positive, negative, or neutral. The existing techniques have a number of problems, namely: they do not take into account the conditions expressed in the opinions (e.g., when they hold and when they do not), they do not generally use any contextual information (e.g., past user opinions on the same attribute), and they are not validated on big datasets (e.g., billions of messages). In this paper, we present Torii, which is an attribute-based polarity analysis technique that takes both conditions and contextual information into account; we also present our approach to validate it on big datasets.
Pearson Rank: A Head-Weighted Gap-Sensitive Score-Based Correlation Coefficient
Authors: Ning Gao, Mossaab Bagdouri, Douglas W. Oard
Venue: SIGIR
Year: 2016

One way of evaluating the reusability of a test collection is to determine whether removing the unique contributions of some system would alter the preference order between that system and others. Rank correlation measures such as Kendall's tau are often used for this purpose. Rank correlation measures are appropriate for ordinal measures in which only preference order is important, but many evaluation measures produce system scores in which both the preference order and the magnitude of the score difference are important. Such measures are referred to as interval. Pearson's rho offers one way in which correlation can be computed over results from an interval measure such that smaller errors in the gap size are preferred. When seeking to improve over existing systems, we care the most about comparisons among the best systems. For that purpose we prefer head-weighed measures such as tau_AP, which is designed for ordinal data. No present head weighted measure fully leverages the information present in interval effectiveness measures. This paper introduces such a measure, referred to as Pearson Rank.
Tracking Sentiment by Time Series Analysis
Authors: Anastasia Giachanou, Fabio Crestani
Venue: SIGIR
Year: 2016

In recent years social media have emerged as popular platforms for people to share their thoughts and opinions on all kind of topics. Tracking opinion over time is a powerful tool that can be used for sentiment prediction or to detect the possible reasons of a sentiment change. Understanding topic and sentiment evolution allows enterprises or government to capture negative sentiment and act promptly. In this study, we explore conventional time series analysis methods and their applicability on topic and sentiment trend analysis. We use data collected from Twitter that span over nine months. Finally, we study the usability of outliers detection and different measures such as sentiment velocity and acceleration on the task of sentiment tracking.
Succinct Data Structures in Information Retrieval: Theory and Practice
Authors: Simon Gog, Rossano Venturini
Venue: SIGIR
Year: 2016

Succinct data structures are used today in many information retrieval applications, e.g., posting lists representation, language model representation, indexing (social) graphs, query auto-completion, document retrieval and indexing dictionary of strings, just to mention the most recent ones. These new kind of data structures mimic the operations of their classical counterparts within a comparable time complexity but require much less space. With the availability of several libraries for basic succinct structures - like SDSL, Succinct, Facebook?s Folly, and Sux - it is relatively easy to directly profit from advances in this field. In this tutorial we will introduce this field of research by presenting the most important succinct data structures to represent set of integers, set of points, trees, graphs and strings together with their most important applications to Information Retrieval problems. The introduction of the succinct data structures will be sustained with a practical session with programming handouts to solve. This will allow the attendees to directly experiment with implementations of these solutions on real datasets and understand the potential benefits they can bring on their own projects.
Online Learning to Rank for Information Retrieval: SIGIR 2016 Tutorial
Authors: Artem Grotov, Maarten de Rijke
Venue: SIGIR
Year: 2016

During the past 10--15 years offline learning to rank has had a tremendous influence on information retrieval, both scientifically and in practice. Recently, as the limitations of offline learning to rank for information retrieval have become apparent, there is increased attention for online learning to rank methods for information retrieval in the community. Such methods learn from user interactions rather than from a set of labeled data that is fully available for training up front.
iGlasses: A Novel Recommendation System for Best-fit Glasses
Authors: Xiaoling Gu, Lidan Shou, Pai Peng, Ke Chen, Sai Wu, Gang Chen
Venue: SIGIR
Year: 2016

We demonstrate iGlasses, a novel recommendation system that accepts a frontal face photo as the input and returns the best-fit eyeglasses as the output. As conventional recommendation techniques such as collaborative filtering become inapplicable in the problem, we propose a new recommendation method which exploits the implicit matching rules between human faces and eyeglasses. We first define fine-grained attributes for human faces and frames of glasses respectively. Then, we develop a recommendation framework based on a probabilistic graphical model, which effectively captures the correlation among these fine-grained attributes. Ranking of the frames (glasses) is done by their similarity to the query facial attributes. Finally, we produce a synthesized image for the input face to demonstrate the visual effect when wearing the recommended glasses.
Searching by Talking: Analysis of Voice Queries on Mobile Web Search
Authors: Ido Guy
Venue: SIGIR
Year: 2016

The growing popularity of mobile search and the advancement in voice recognition technologies have opened the door for web search users to speak their queries, rather than type them. While this kind of voice search is still in its infancy, it is gradually becoming more widespread. In this paper, we examine the logs of a commercial search engine's mobile interface, and compare the spoken queries to the typed-in queries. We place special emphasis on the semantic and syntactic characteristics of the two types of queries. %Our analysis suggests that voice queries focus more on audio-visual content and question answering, and less on social networking and adult domains. We also conduct an empirical evaluation showing that the language of voice queries is closer to natural language than typed queries. Our analysis reveals further differences between voice and text search, which have implications for the design of future voice-enabled search tools.