OpenRaiser/Intern-Atlas huggingface dataset:
Fast document-at-a-time query processing using two-tier indexes
Authors: Cristian Rossi, Edleno Silva de Moura, André Luiz da Costa Carvalho, Altigran Soares da Silva
Venue: SIGIR
Year: 2013

In this paper we present two new algorithms designed to reduce the overall time required to process top-k queries. These algorithms are based on the document-at-a-time approach and modify the best baseline we found in the literature, Blockmax WAND (BMW), to take advantage of a two-tiered index, in which the first tier is a small index containing only the higher impact entries of each inverted list. This small index is used to pre-process the query before accessing a larger index in the second tier, resulting in considerable speeding up the whole process. The first algorithm we propose, named BMW-CS, achieves higher performance, but may result in small changes in the top results provided in the final ranking. The second algorithm, named BMW-t, preserves the top results and, while slower than BMW-CS, it is faster than BMW. In our experiments, BMW-CS was more than 40 times faster than BMW when computing top 10 results, and, while it does not guarantee preserving the top results, it preserved all ranking results evaluated at this level.
Composition of TF normalizations: new insights on scoring functions for ad hoc IR
Authors: François Rousseau, Michalis Vazirgiannis
Venue: SIGIR
Year: 2013

Previous papers in ad hoc IR reported that scoring functions should satisfy a set of heuristic retrieval constraints, providing a mathematical justification for the normalizations historically applied to the term frequency (TF). In this paper, we propose a further level of abstraction, claiming that the successive normalizations are carried out through composition. Thus we introduce a principled framework that fully explains BM25 as a variant of TF-IDF with an inverse order of function composition. Our experiments over standard datasets indicate that the respective orders of composition chosen in the original papers for both TF-IDF and BM25 are the most effective ones. Moreover, since the order is different between the two models, they also demonstrated that the order is instrumental in the design of weighting models. In fact, while considering more complex scoring functions such as BM25+, we discovered a novel weighting model in terms of order of composition that consistently outperforms all the rest. Our contribution here is twofold: we provide a unifying mathematical framework for IR and a novel scoring function discovered using this framework.
A portable multilingual medical directory by automatic categorization of Wikipedia articles
Authors: Fernando Ruiz-Rico, María-Consuelo Rubio-Sánchez, David Tomás, José Luis Vicedo González
Venue: SIGIR
Year: 2013

Wikipedia has become one of the most important sources of information available all over the world. However, the categorization of Wikipedia articles is not standardized and the searches are mainly performed on keywords rather than concepts. In this paper we present an application that builds a hierarchical structure to organize all Wikipedia entries, so that medical articles can be reached from general to particular, using the well known Medical Subject Headings (MeSH) thesaurus. Moreover, the language links between articles will allow using the directory created in different languages. The final system can be packed and ported to mobile devices as a standalone offline application.
A multilingual and multiplatform application for medicinal plants prescription from medical symptoms
Authors: Fernando Ruiz-Rico, David Tomás, José Luis Vicedo González, María-Consuelo Rubio-Sánchez
Venue: SIGIR
Year: 2013

This paper presents an application for medicinal plants prescription based on text classification techniques. The system receives as an input a free text describing the symptoms of a user, and retrieves a ranked list of medicinal plants related to those symptoms. In addition, a set of links to Wikipedia are also provided, enriching the information about every medicinal plant presented to the user. In order to improve the accessibility to the application, the input can be written in six different languages, adapting the results accordingly. The application interface can be accessed from different devices and platforms.
Designing search usability
Authors: Tony Russell-Rose
Venue: SIGIR
Year: 2013

Search is not just a box and ten blue links. Search is a journey: an exploration where what we encounter along the way changes what we seek. But in order to guide people along this journey, we must understand both the art and science of search experience design. The aim of this tutorial is to deliver a course grounded in good scholarship, integrating the latest research findings with insights derived from the practical experience of designing and optimizing an extensive range of commercial search applications. It focuses on the development of transferable, practical skills that can be learnt and practiced within a half-day session.
The impact of intent selection on diversified search evaluation
Authors: Tetsuya Sakai, Zhicheng Dou, Charles L. A. Clarke
Venue: SIGIR
Year: 2013

To construct a diversified search test collection, a set of possible subtopics (or intents) needs to be determined for each topic, in one way or another, and perintent relevance assessments need to be obtained. In the TREC Web Track Diversity Task, subtopics are manually developed at NIST, based on results of automatic click log analysis; in the NTCIR INTENT Task, intents are determined by manually clustering 'subtopics strings' returned by participating systems. In this study, we address the following research question: Does the choice of intents for a test collection affect relative performances of diversified search systems? To this end, we use the TREC 2012 Web Track Diversity Task data and the NTCIR-10 INTENT-2 Task data, which share a set of 50 topics but have different intent sets. Our initial results suggest that the choice of intents may affect relative performances, and that this choice may be far more important than how many intents are selected for each topic
Summary of the NTCIR-10 INTENT-2 task: subtopic mining and search result diversification
Authors: Tetsuya Sakai, Zhicheng Dou, Takehiro Yamamoto, Yiqun Liu, Min Zhang, Makoto P. Kato, Ruihua Song, Mayu Iwata
Venue: SIGIR
Year: 2013

The NTCIR INTENT task comprises two subtasks: {\em Subtopic Mining}, where systems are required to return a ranked list of {\em subtopic strings} for each given query; and {\em Document Ranking}, where systems are required to return a diversified web search result for each given query. This paper summarises the novel features of the Second INTENT task at NTCIR-10 and its main findings, and poses some questions for future diversified search evaluation.
The role of current working context in professional search
Authors: Maya Sappelli
Venue: SIGIR
Year: 2013

Today's working world of knowledge workers is changing rapidly. The available information that they need to process is ever growing. In addition, the characteristics of their work are changing as people can and do their work from home. This has resulted in the need to support knowledge workers in order to prevent burnouts. The project SWELL (http://www.swell-project.net) targets this by developing systems that support user's mental and physical well-being at work and at home. In the PhD project presented in this abstract we aim at maintaining well-being at work through information support.
Search engine switching detection based on user personal preferences and behavior patterns
Authors: Denis Savenkov, Dmitry Lagun, Qiaoling Liu
Venue: SIGIR
Year: 2013

Sometimes, during a search task users may switch from one search engine to another for several reasons, e.g., dissatisfaction with the current search results or desire for broader topic coverage. Detecting the fact of switching is difficult but important for understanding users' satisfaction with the search engine and the complexity of their search tasks, leading to economic significance for search providers. Previous research on switching detection mainly focused on studying different signals useful for the task and particular reasons for switching. Although it is known that switching is a personal choice of a user and different users have different search behavior, little has been done to understand how these differences could be used for switching detection. In this paper we study the effectiveness of learning personal behavior patterns for switching detection and present a personalized approach which uses user's session history containing sessions with and without switches. Experiments show that users' personal habits and behavior patterns are indeed among the most informative signals. Our findings can be used by a search log analyzer for engine switching detection and potentially other log mining problems, thus providing valuable signals for search providers to improve user experience.
RecSys for distributed events: investigating the influence of recommendations on visitor plans
Authors: Richard Schaller, Morgan Harvey, David Elsweiler
Venue: SIGIR
Year: 2013

Distributed events are collections of events taking place within a small area over the same time period and relating to a single topic. There are often a large number of events on offer and the times in which they can be visited are heavily constrained, therefore the task of choosing events to visit and in which order can be very difficult. In this work we investigate how visitors can be assisted by means of a recommender system via 2 large-scale naturalistic studies (n=860 and n=1047). We show that a recommender system can influence users to select events that result in tighter and more compact routes, thus allowing users to spend less time travelling and more time visiting events.
Hybrid retrieval approaches to geospatial music recommendation
Authors: Markus Schedl, Dominik Schnitzer
Venue: SIGIR
Year: 2013

Recent advances in music retrieval and recommendation algorithms highlight the necessity to follow multimodal approaches in order to transcend limits imposed by methods that solely use audio, web, or collaborative filtering data. In this paper, we propose hybrid music recommendation algorithms that combine information on the music content, the music context, and the user context, in particular, integrating location-aware weighting of similarities. Using state-of-the-art techniques to extract audio features and contextual web features, and a novel standardized data set of music listening activities inferred from microblogs (MusicMicro), we propose several multimodal retrieval functions.
The effect of threshold priming and need for cognition on relevance calibration and assessment
Authors: Falk Scholer, Diane Kelly, Wan-Ching Wu, Hanseul S. Lee, William Webber
Venue: SIGIR
Year: 2013

Human assessments of document relevance are needed for the construction of test collections, for ad-hoc evaluation, and for training text classifiers. Showing documents to assessors in different orderings, however, may lead to different assessment outcomes. We examine the effect that \defineterm{threshold priming}, seeing varying degrees of relevant documents, has on people's calibration of relevance. Participants judged the relevance of a prologue of documents containing highly relevant, moderately relevant, or non-relevant ocuments, followed by a common epilogue of documents of mixed relevance. We observe that participants exposed to only non-relevant documents in the prologue assigned significantly higher average relevance scores to prologue and epilogue documents than participants exposed to moderately or highly relevant documents in the prologue. We also examine how \defineterm{need for cognition}, an individual difference measure of the extent to which a person enjoys engaging in effortful cognitive activity, impacts relevance assessments. High need for cognition participants had a significantly higher level of agreement with expert assessors than low need for cognition participants did. Our findings indicate that assessors should be exposed to documents from multiple relevance levels early in the judging process, in order to calibrate their relevance thresholds in a balanced way, and that individual difference measures might be a useful way to screen assessors.
Finding impressive social content creators: searching for SNS illustrators using feedback on motifs and impressions
Authors: Yohei Seki, Kiyoto Miyajima
Venue: SIGIR
Year: 2013

We propose a method for finding impressive creators in online social network sites (SNSs). Many users are actively engaged in publishing their own works, sharing visual content on sites such as YouTube or Flickr. In this paper, we focus on the Japanese illustration-sharing SNS, Pixiv. We implement an illustrator search system based on user impression categories. The impressions of illustrators are estimated from clues in the crowdsourced social-tag annotations on their illustrations. We evaluated our system in terms of normalized discounted cumulative gain and found that using feedback on motifs and impressions for illustrations of relevant illustrators improved illustrator search by 11%.
Displaying relevance scores for search results
Authors: Guy Shani, Noam Tractinsky
Venue: SIGIR
Year: 2013

Internet search engines typically compute a relevance score for webpages given the query terms, and then rank the pages by decreasing relevance scores. The popular search engines do not, however, present the relevance scores that were computed during this process. We suggest that these relevance scores may contain information that can help users make conscious decisions. In this paper we evaluate in a user study how users react to the display of such scores. The results indicate that users understand graphical displays of relevance, and make decisions based on these scores. Our results suggest that in the context of exploratory search, relevance scores may cause users to explore more search results.
Competence-based song recommendation
Authors: Lidan Shou, Kuang Mao, Xinyuan Luo, Ke Chen, Gang Chen, Tianlei Hu
Venue: SIGIR
Year: 2013

Singing is a popular social activity and a good way of expressing one's feelings. One important reason for unsuccessful singing performance is because the singer fails to choose a suitable song. In this paper, we propose a novel singing competence-based song recommendation framework. It is distinguished from most existing music recommendation systems which rely on the computation of listeners' interests or similarity. We model a singer's vocal competence as singer profile, which takes voice pitch, intensity, and quality into consideration. Then we propose techniques to acquire singer profiles. We also present a song profile model which is used to construct a human annotated song database. Finally, we propose a learning-to-rank scheme for recommending songs by singer profile. The experimental study on real singers demonstrates the effectiveness of our approach and its advantages over two baseline methods. To the best of our knowledge, our work is the first to study competence-based song recommendation.
Riding the multimedia big data wave
Authors: John R. Smith
Venue: SIGIR
Year: 2013

In this talk we present a perspective across multiple industry problems, including safety and security, medical, Web, social and mobile media, and motivate the need for large-scale analysis and retrieval of multimedia data. We describe a multi-layer architecture that incorporates capabilities for audio-visual feature extraction, machine learning and semantic modeling and provides a powerful framework for learning and classifying contents of multimedia data. We discuss the role semantic ontologies for representing audio-visual concepts and relationships, which are essential for training semantic classifiers. We discuss the importance of using faceted classification schemes in particular for organizing multimedia semantic concepts in order to achieve effective learning and retrieval. We also show how training and scoring of multimedia semantics can be implemented on big data distributed computing platforms to address both massive-scale analysis and low-latency processing. We describe multiple efforts at IBM on image and video analysis and retrieval, including IBM Multimedia Analysis and Retrieval System (IMARS), and show recent results for semantic-based classification and retrieval. We conclude with future directions for improving analysis of multimedia through interactive and curriculum-based techniques for multimedia semantics-based learning and retrieval.
Building test collections: an interactive tutorial for students and others without their own evaluation conference series
Authors: Ian Soboroff
Venue: SIGIR
Year: 2013

While existing test collections and evaluation conference efforts may sufficiently support one's research, one can easily find oneself wanting to solve problems no one else is solving yet. But how can research in IR be done (or be published!) without solid data and experiments? Not everyone can talk TREC, CLEF, INEX, or NTCIR into running a track to build a collection.
Estimating query representativeness for query-performance prediction
Authors: Mor Sondak, Anna Shtok, Oren Kurland
Venue: SIGIR
Year: 2013

The query-performance prediction (QPP) task is estimating retrieval effectiveness with no relevance judgments. We present a novel probabilistic framework for QPP that gives rise to an important aspect that was not addressed in previous work; namely, the extent to which the query effectively represents the information need for retrieval. Accordingly, we devise a few query-representativeness measures that utilize relevance language models. Experiments show that integrating the most effective measures with state-of-the-art predictors in our framework often yields prediction quality that significantly transcends that of using the predictors alone.
Modeling term dependencies with quantum language models for IR
Authors: Alessandro Sordoni, Jian-Yun Nie, Yoshua Bengio
Venue: SIGIR
Year: 2013

Traditional information retrieval (IR) models use bag-of-words as the basic representation and assume that some form of independence holds between terms. Representing term dependencies and defining a scoring function capable of integrating such additional evidence is theoretically and practically challenging. Recently, Quantum Theory (QT) has been proposed as a possible, more general framework for IR. However, only a limited number of investigations have been made and the potential of QT has not been fully explored and tested. We develop a new, generalized Language Modeling approach for IR by adopting the probabilistic framework of QT. In particular, quantum probability could account for both single and compound terms at once without having to extend the term space artificially as in previous studies. This naturally allows us to avoid the weight-normalization problem, which arises in the current practice by mixing scores from matching compound terms and from matching single terms. Our model is the first practical application of quantum probability to show significant improvements over a robust bag-of-words baseline and achieves better performance on a stronger non bag-of-words baseline.
Faster upper bounding of intersection sizes
Authors: Daisuke Takuma, Hiroki Yanagisawa
Venue: SIGIR
Year: 2013

There is a long history of developing efficient algorithms for set intersection, which is a fundamental operation in information retrieval and databases. In this paper, we describe a new data structure, a Cardinality Filter, to quickly compute an upper bound on the size of a set intersection. Knowing an upper bound of the size can be used to accelerate many applications such as top-k query processing in text mining. Given finite sets A and B, the expected computation time for the upper bound of the size of the intersection |A cap B| is O( (|A| + |B|) w), where w is the machine word length. This is much faster than the current best algorithm for the exact intersection, which runs in O((|A| + |B|) / √w + |A cap B|) expected time. Our performance studies show that our implementations of Cardinality Filters are from 2 to 10 times faster than existing set intersection algorithms, and the time for a top-k query in a text mining application can be reduced by half.
Studying page life patterns in dynamical web
Authors: Alexey Tikhonov, Ivan Bogatyy, Pavel Burangulov, Liudmila Ostroumova, Vitaliy Koshelev, Gleb Gusev
Venue: SIGIR
Year: 2013

With the ever-increasing speed of content turnover on the web, it is particularly important to understand the patterns that pages' popularity follows. This paper focuses on the dynamical part of the web, i.e. pages that have a limited lifespan and experience a short popularity outburst within it. We classify these pages into five patterns based on how quickly they gain popularity and how quickly they lose it. We study the properties of pages that belong to each pattern and determine content topics that contain disproportionately high fractions of particular patterns. These developments are utilized to create an algorithm that approximates with reasonable accuracy the expected popularity pattern of a web page based on its URL and, if available, prior knowledge about its domain's topics.
Flat vs. hierarchical phrase-based translation models for cross-language information retrieval
Authors: Ferhan Türe, Jimmy Lin
Venue: SIGIR
Year: 2013

Although context-independent word-based approaches remain popular for cross-language information retrieval, many recent studies have shown that integrating insights from modern statistical machine translation systems can lead to substantial improvements in effectiveness. In this paper, we compare flat and hierarchical phrase-based translation models for query translation. Both approaches yield significantly better results than either a token-based or a one-best translation baseline on standard test collections. The choice of model manifests interesting tradeoffs in terms of effectiveness, efficiency, and model compactness.
A comparison of the optimality of statistical significance tests for information retrieval evaluation
Authors: Julián Urbano, Mónica Marrero, Diego Martín
Venue: SIGIR
Year: 2013

Previous research has suggested the permutation test as the theoretically optimal statistical significance test for IR evaluation, and advocated for the discontinuation of the Wilcoxon and sign tests. We present a large-scale study comprising nearly 60 million system comparisons showing that in practice the bootstrap, t-test and Wilcoxon test outperform the permutation test under different optimality criteria. We also show that actual error rates seem to be lower than the theoretically expected 5%, further confirming that we may actually be underestimating significance.
Spacious: an interactive mental search interface
Authors: Phong D. Vo, Hichem Sahbi
Venue: SIGIR
Year: 2013

We introduce in this work a novel approach for semantic indexing and mental image search. Given semantic concepts defined by few training examples, our formulation is transductive and learns a mapping from an initial ambient space, related to low level visual features, to an output space spanned by a well defined semantic basis where data can be easily explored. With this method, searching for a mental visual target reduces to scanning data according to their coordinates in the learned semantic space. We illustrate the proposed method through our graphical user interface "Spacious", for the purpose of visualization and interactive navigation in generic image databases and satellite images.