OpenRaiser/Intern-Atlas huggingface dataset:
Using the cross-entropy method to re-rank search results
Authors: Haggai Roitman, Shay Hummel, Oren Kurland
Venue: SIGIR
Year: 2014

We present a novel unsupervised approach to re-ranking an initially retrieved list. The approach is based on the Cross Entropy method applied to permutations of the list, and relies on performance prediction. Using pseudo predictors we establish a lower bound on the prediction quality that is required so as to have our approach significantly outperform the original retrieval. Our experiments serve as a proof of concept demonstrating the considerable potential of the proposed approach. A case in point, only a tiny fraction of the huge space of permutations needs to be explored to attain significant improvements over the original retrieval.
A fusion approach to cluster labeling
Authors: Haggai Roitman, Shay Hummel, Michal Shmueli-Scheuer
Venue: SIGIR
Year: 2014

We present a novel approach to the cluster labeling task using fusion methods. The core idea of our approach is to weigh labels, suggested by any labeler, according to the estimated labeler's decisiveness with respect to each of its suggested labels. We hypothesize that, a cluster labeler's labeling choice for a given cluster should remain stable even in the presence of a slightly incomplete cluster data. Using state-of-the-art cluster labeling and data fusion methods, evaluated over a large data collection of clusters, we demonstrate that, overall, the cluster labeling fusion methods that further consider the labeler's decisiveness provide the best labeling performance.
Recommending social media content to community owners
Authors: Inbal Ronen, Ido Guy, Elad Kravi, Maya Barnea
Venue: SIGIR
Year: 2014

Online communities within the enterprise offer their leaders an easy and accessible way to attract, engage, and influence others. Our research studies the recommendation of social media content to leaders (owners) of online communities within the enterprise. We developed a system that suggests to owners new content from outside the community, which might interest the community members. As online communities are taking a central role in the pervasion of social media to the enterprise, sharing such recommendations can help owners create a more lively and engaging community. We compared seven different methods for generating recommendations, including content-based, member-based, and hybridization of the two. For member-based recommendations, we experimented with three groups: owners, active members, and regular members. Our evaluation is based on a survey in which 851 community owners rated a total of 8,218 recommended content items. We analyzed the quality of the different recommendation methods and examined the effect of different community characteristics, such as type and size.
Improving unsupervised query segmentation using parts-of-speech sequence information
Authors: Rishiraj Saha Roy, Yogarshi Vyas, Niloy Ganguly, Monojit Choudhury
Venue: SIGIR
Year: 2014

We present a generic method for augmenting unsupervised query segmentation by incorporating Parts-of-Speech (POS) sequence information to detect meaningful but rare n-grams. Our initial experiments with an existing English POS tagger employing two different POS tagsets and an unsupervised POS induction technique specifically adapted for queries show that POS information can significantly improve query segmentation performance in all these cases.
ExperTime: tracking expertise over time
Authors: Jan Rybak, Krisztian Balog, Kjetil Nørvåg
Venue: SIGIR
Year: 2014

This paper presents ExperTime, a web-based system for tracking expertise over time. We visualize a person's expertise profile on a timeline, where we detect and characterize changes in the focus or topics of expertise. It is possible to zoom in on a given time period in order to examine the underlying data that is used as supporting evidence. It is also possible to perform visual and quantitative comparison of two arbitrarily selected time periods in a highly interactive environment. We invite profile owners to evaluate and fine-tune their profiles, and to leave feedback.
Uncovering the unarchived web
Authors: Thaer Samar, Hugo C. Huurdeman, Anat Ben-David, Jaap Kamps, Arjen P. de Vries
Venue: SIGIR
Year: 2014

Many national and international heritage institutes realize the importance of archiving the web for future culture heritage. Web archiving is currently performed either by harvesting a national domain, or by crawling a pre-defined list of websites selected by the archiving institution. In either method, crawling results in more information being harvested than just the websites intended for preservation; which could be used to reconstruct impressions of pages that existed on the live web of the crawl date, but would have been lost forever. We present a method to create representations of what we will refer to as a web collection's (aura): the web documents that were not included in the archived collection, but are known to have existed --- due to their mentions on pages that were included in the archived web collection. To create representations of these unarchived pages, we exploit the information about the unarchived URLs that can be derived from the crawls by combining crawl date distribution, anchor text and link structure. We illustrate empirically that the size of the aura can be substantial: in 2012, the Dutch Web archive contained 12.3M unique pages, while we uncover references to 11.9M additional (unarchived) pages.
To hint or not: exploring the effectiveness of search hints for complex informational tasks
Authors: Denis Savenkov, Eugene Agichtein
Venue: SIGIR
Year: 2014

Extensive previous research has shown that searchers often require assistance with query formulation and refinement. Yet, it is not clear what kind of assistance is most useful, and how effective it is both objectively (e.g., in terms of task success) and subjectively (e.g., in terms of searcher percep- tion of the search difficulty). This work describes the results of a controlled user study comparing the effects of provid- ing specific vs. generic search hints on search success and satisfaction. Our results indicate that specific search hints tend to effectively improve searcher success rates and reduce perceived effort, while generic ones can be detrimental in both search effectiveness and user satisfaction. The results of this study are an important step towards the design of future search systems that could effectively assist and guide the user in accomplishing complex search tasks.
SoMeRA 2014: social media retrieval and analysis workshop
Authors: Markus Schedl, Peter Knees, Jialie Shen
Venue: SIGIR
Year: 2014

The SoMeRA workshop targets cutting edge research from all fields of retrieval, recommendation, and browsing in social media, as well as the analysis of user's multifaceted traces therein. Submissions to the workshop cover a broad range of topics including multimedia retrieval and exploration, user-aware recommender systems, network analysis, event detection, and computational linguistics.
User geospatial context for music recommendation in microblogs
Authors: Markus Schedl, Andreu Vall, Katayoun Farrahi
Venue: SIGIR
Year: 2014

Music information retrieval and music recommendation are seeing a paradigm shift towards methods that incorporate user context aspects. However, structured experiments on a standardized music dataset to investigate the effects of doing so are scarce. In this paper, we compare performance of various combinations of collaborative filtering and geospatial as well as cultural user models for the task of music recommendation. To this end, we propose a geospatial model that uses GPS coordinates and a cultural model that uses semantic locations (continent, country, and state of the user). We conduct experiments on a novel standardized music collection, the ``Million Musical Tweets Dataset'' of listening events extracted from microblogs. Overall, we find that modeling listeners' location via Gaussian mixture models and computing similarities from these outperforms both cultural user models and collaborative filtering.
Interactions between health searchers and search engines
Authors: Georg P. Schoenherr, Ryen W. White
Venue: SIGIR
Year: 2014

The Web is an important resource for understanding and diagnosing medical conditions. Based on exposure to online content, people may develop undue health concerns, believ- ing that common and benign symptoms are explained by se- rious illnesses. In this paper, we investigate potential strate- gies to mine queries and searcher histories for clues that could help search engines choose the most appropriate infor- mation to present in response to exploratory medical queries. To do this, we performed a longitudinal study of health search behavior using the logs of a popular Web search en- gine. We found that query variations which might appear innocuous (e.g. "bad headache" vs "severe headache") may hold valuable information about the searcher which could be used by search engines to improve performance. Fur- thermore, we investigated how medically-concerned users re- spond differently to search engine result pages (SERPs) and find that their disposition for clicking on concerning pages is pronounced, potentially leading to a self-reinforcement of concern. Finally, we studied to which degree variations in the SERP impact future search and real-world health- seeking behavior and obtained some surprising results (e.g., viewing concerning pages may lead to a short-term reduction of in-world healthcare utilization).
PRISM: concept-preserving social image search results summarization
Authors: Boon-Siew Seah, Sourav S. Bhowmick, Aixin Sun
Venue: SIGIR
Year: 2014

Most existing tag-based social image search engines present search results as a ranked list of images, which cannot be consumed by users in a natural and intuitive manner. In this paper, we present a novel concept-preserving image search results summarization algorithm named Prism. Prism exploits both visual features and tags of the search results to generate high quality summary, which not only breaks the results into visually and semantically coherent clusters but it also maximizes the coverage of the summary w.r.t the original search results. It first constructs a visual similarity graph where the nodes are images in the search results and the edges represent visual similarities between pairs of images. This graph is optimally decomposed and compressed into a set of concept-preserving subgraphs based on a set of summarization objectives. Images in a concept-preserving subgraph are visually and semantically cohesive and are described by a minimal set of tags or concepts. Lastly, one or more exemplar images from each subgraph is selected to form the exemplar summary of the result set. Through empirical study, we demonstrate the effectiveness of Prism against state-of-the-art image summarization and clustering algorithms.
Cluster links prediction for literature based discovery using latent structure and semantic features
Authors: Yakub Sebastian
Venue: SIGIR
Year: 2014

The potential impact of a scientific article has a significant correlation with its ability to establish novel connections between pre-existing knowledge [1-2]. Discovering hidden connections between the existing scientific literature is an interesting yet highly challenging information retrieval problem [2]. Literature based discovery (LBD) uses computational algorithms to discover potential hidden connections between previously disconnected sets of literature [3]. Most of the current LBD methods focus on analyzing latent semantic features in texts but are usually computationally demanding. In particular, they do not aim at predicting novel discovery links between clusters of literature.
Privacy-preserving IR: when information retrieval meets privacy and security
Authors: Luo Si, Hui Yang
Venue: SIGIR
Year: 2014

Information retrieval (IR) and information privacy/security are two fast-growing computer science disciplines. There are many synergies and connections between these two disciplines. However, there have been very limited efforts to connect the two important disciplines. On the other hand, due to lack of mature techniques in privacy-preserving IR, concerns about information privacy and security have become serious obstacles that prevent valuable user data to be used in IR research such as studies on query logs, social media, tweets, sessions, and medical record retrieval. This privacy-preserving IR workshop aims to spur research that brings together the research fields of IR and privacy/security, and research that mitigates privacy threats in information retrieval by constructing novel algorithms and tools that enable web users to better understand associated privacy risks.
Enhancing personalization via search activity attribution
Authors: Adish Singla, Ryen W. White, Ahmed Hassan Awadallah, Eric Horvitz
Venue: SIGIR
Year: 2014

Online services rely on machine identifiers to tailor services such as personalized search and advertising to individual users. The assumption made is that each identifier comprises the behavior of a single person. However, shared machine usage is common, and in these cases, the activities of multiple users may be generated under a single identifier, creating a potentially noisy signal for applications such as search personalization. We propose enhancing Web search personalization with methods that can disambiguate among different users of a machine, thus connecting the current query with the appropriate search history. Using logs containing both person and machine identifiers, and logs from a popular commercial search engine, we learn models that accurately assign observed search behaviors to each of different users. This information is then used to augment existing personalization methods that are currently based only on machine identifiers. We show that this new capability to infer users can be used to improve the performance of existing personalization methods. The early findings of our research are promising and have implications for search personalization.
Mouse movement during relevance judging: implications for determining user attention
Authors: Mark D. Smucker, Xiaoyu Sunny Guo, Andrew Toulis
Venue: SIGIR
Year: 2014

Several researchers have found that a user's mouse position gives an indication of the user's gaze during web search and other tasks. As part of a user study that involved relevance judging of document summaries and full documents, we recorded users' mouse movements. We found that in a large number of cases, the users did nothing more with their mouse than move it to the buttons used for recording the relevance decision. In addition, we found that different search topics can result in large differences in the amount of mouse movement that is indicative of user attention. For simple reading tasks, such as short document summaries, mouse-tracking does not appear to be an effective means of discerning user attention. While more complex tasks may allow mouse movements to provide information regarding user attention, on average, indications of user attention existed in only 59% of the relevance judgments made for full documents.
Learning to translate queries for CLIR
Authors: Artem Sokolov, Felix Hieber, Stefan Riezler
Venue: SIGIR
Year: 2014

The statistical machine translation (SMT) component of cross-lingual information retrieval (CLIR) systems is often regarded as black box that is optimized for translation quality independent from the retrieval task. In recent work [10], SMT has been tuned for retrieval by training a reranker on $k$-best translations ordered according to their retrieval performance. In this paper we propose a decomposable proxy for retrieval quality that obviates the need for costly intermediate retrieval. Furthermore, we explore the full search space of the SMT decoder by directly optimizing decoder parameters under a retrieval-based objective. Experimental results for patent retrieval show our approach to be a promising alternative to the standard pipeline approach.
Enrichment of user profiles across multiple online social networks for volunteerism matching for social enterprise
Authors: Xuemeng Song
Venue: SIGIR
Year: 2014

Volunteers are extremely crucial to nonprofit organizations (NPOs) to sustain their continuing operations. On the other hand, many talents are looking for appropriate volunteer opportunities to realize their dreams of making an impact on the world with their expertise. This is a typical supply and demand matching issue. Fortunately, user profiling and the discovery of user volunteering tendency can benefit from users' continuous enthusiasm and active participation in diverse online social networks (OSNs) and the huge amount of publicly available user generated contents (UGCs). In this work, we aim to bridge the gap between the supply of talents with volunteering tendency and the demands of social enterprise and enhance the social welfare. This is done by incorporating volunteering tendency into user profiling across multiple OSNs. Consequently, this interdisciplinary research opens a new window for both computer science and social science. To the best of our knowledge, this is the first attempt to tackle the problem of volunteer matching for social enterprise based on publicly available UGCs. First, we explain the definitions of the main concepts with examples. Second, we propose a system architecture for addressing the problem of volunteerism matching that Includes three components: Profile Collection, Profile Enrichment and Profile Matching. Finally, we identify the major challenges encountered in our current research work. This paper discusses our design and progress in this research.
Context-aware web search abandonment prediction
Authors: Yang Song, Xiaolin Shi, Ryen White, Ahmed Hassan Awadallah
Venue: SIGIR
Year: 2014

Web search queries without hyperlink clicks are often referred to as abandoned queries. Understanding the reasons for abandonment is crucial for search engines in evaluating their performance. Abandonment can be categorized as good or bad depending on whether user information needs are satisfied by result page content. Previous research has sought to understand abandonment rationales via user surveys, or has developed models to predict those rationales using behavioral patterns. However, these models ignore important contextual factors such as the relationship between the abandoned query and prior abandonment instances. We propose more advanced methods for modeling and predicting abandonment rationales using contextual information from user search sessions by analyzing search engine logs, and discover dependencies between abandoned queries and user behaviors. We leverage these dependency signals to build a sequential classifier using a structured learning framework designed to handle such signals. Our experimental results show that our approach is 22% more accurate than the state-of-the-art abandonment-rationale classifier. Going beyond prediction, we leverage the prediction results to significantly improve relevance using instances of predicted good and bad abandonment.
User-driven system-mediated collaborative information retrieval
Authors: Laure Soulier, Chirag Shah, Lynda Tamine
Venue: SIGIR
Year: 2014

Most of the previous approaches surrounding collaborative information retrieval (CIR) provide either a user-based mediation, in which the system only supports users' collaborative activities, or a system-based mediation, in which the system plays an active part in balancing user roles, re-ranking results, and distributing them to optimize overall retrieval performance. In this paper, we propose to combine both of these approaches by a role mining methodology that learns from users' actions about the retrieval strategy they adapt. This hybrid method aims at showing how users are different and how to use these differences for suggesting roles. The core of the method is expressed as an algorithm that (1) monitors users' actions in a CIR setting; (2) discovers differences among the collaborators along certain dimensions; and (3) suggests appropriate roles to make the most out of individual skills and optimize IR performance. Our approach is empirically evaluated and relies on two different laboratory studies involving 70 pairs of users. Our experiments show promising results that highlight how role mining could optimize the collaboration within a search session. The contributions of this work include a new algorithm for mining user roles in collaborative IR, an evaluation methodology, and a new approach to improve IR performance with the operationalization of user-driven system-mediated collaboration.
Learning similarity functions for topic detection in online reputation monitoring
Authors: Damiano Spina, Julio Gonzalo, Enrique Amigó
Venue: SIGIR
Year: 2014

Reputation management experts have to monitor--among others--Twitter constantly and decide, at any given time, what is being said about the entity of interest (a company, organization, personality...). Solving this reputation monitoring problem automatically as a topic detection task is both essential--manual processing of data is either costly or prohibitive--and challenging--topics of interest for reputation monitoring are usually fine-grained and suffer from data sparsity. We focus on a solution for the problem that (i) learns a pairwise tweet similarity function from previously annotated data, using all kinds of content-based and Twitter-based features; (ii) applies a clustering algorithm on the previously learned similarity function. Our experiments indicate that (i) Twitter signals can be used to improve the topic detection process with respect to using content signals only; (ii) learning a similarity function is a flexible and efficient way of introducing supervision in the topic detection clustering process. The performance of our best system is substantially better than state-of-the-art approaches and gets close to the inter-annotator agreement rate. A detailed qualitative inspection of the data further reveals two types of topics detected by reputation experts: reputation alerts / issues (which usually spike in time) and organizational topics (which are usually stable across time).
Circumlocution in diagnostic medical queries
Authors: Isabelle Stanton, Samuel Ieong, Nina Mishra
Venue: SIGIR
Year: 2014

Circumlocution is when many words are used to describe what could be said with fewer, e.g., "a machine that takes moisture out of the air" instead of "dehumidifier." Web search is a perfect backdrop for circumlocution where people struggle to name what they seek. In some domains, not knowing the correct term can have a significant impact on the search results that are retrieved. We study the medical domain, where professional medical terms are not commonly known and where the consequence of not knowing the correct term can impact the accuracy of surfaced information, as well as escalation of anxiety, and ultimately the medical care sought. Given a free-form colloquial health search query, our objective is to find the underlying professional medical term. The problem is complicated by the fact that people issue quite varied queries to describe what they have. Machine-learning algorithms can be brought to bear on the problem, but there are two key complexities: creating high-quality training data and identifying predictive features. To our knowledge, no prior work has been able to crack this important problem due to the lack of training data. We give novel solutions and demonstrate their efficacy via extensive experiments, greatly improving over the prior art.