OpenRaiser/Intern-Atlas huggingface dataset:
Exploiting term dependence while handling negation in medical search
Authors: Nut Limsopatham, Craig Macdonald, Richard McCreadie, Iadh Ounis
Venue: SIGIR
Year: 2012

In medical records, negative qualifiers, e.g. no or without, are commonly used by health practitioners to identify the absence of a medical condition. Without considering whether the term occurs in a negative or positive context, the sole presence of a query term in a medical record is insufficient to imply that the record is relevant to the query. In this paper, we show how to effectively handle such negation within a medical records information retrieval system. In particular, we propose a term representation that tackles negated language in medical records, which is further extended by considering the dependence of negated query terms. We evaluate our negation handling technique within the search task provided by the TREC Medical Records 2011 track. Our results, which show a significant improvement upon a system that does not consider negated context within records, attest the importance of handling negation.
Cluster-based one-class ensemble for classification problems in information retrieval
Authors: Nedim Lipka, Benno Stein, Maik Anderka
Venue: SIGIR
Year: 2012

A number of relevant information retrieval classification problems are one-class classification problems at heart. I.e., labeled data is only available for one class, the so-called target class, and common discrimination-based classification approaches, be them binary or multiclass, are not applicable. Achieving a high effectiveness when solving one-class problems is difficult anyway and it becomes even more challenging when the target class data is multimodal, which is often the case. To address these concerns we propose a cluster-based one-class ensemble that consists of four steps: (1) applying a clustering algorithm to the target class data, (2) training an individual one-class classifier for each of the identified clusters, (3) aggregating the decisions of the individual classifiers, and (4) selecting the best fitting clustering model. We evaluate our approach with four datasets: an artificially generated dataset, a dataset compiled from a known multiclass text corpus, and two datasets related to one-class problems that received much attention recently, namely authorship verification and quality flaw prediction. Our approach outperforms a one-class SVM on all four datasets.
When web search fails, searchers become askers: understanding the transition
Authors: Qiaoling Liu, Eugene Agichtein, Gideon Dror, Yoelle Maarek, Idan Szpektor
Venue: SIGIR
Year: 2012

While Web search has become increasingly effective over the last decade, for many users' needs the required answers may be spread across many documents, or may not exist on the Web at all. Yet, many of these needs could be addressed by asking people via popular Community Question Answering (CQA) services, such as Baidu Knows, Quora, or Yahoo! Answers. In this paper, we perform the first large-scale analysis of how searchers become askers. For this, we study the logs of a major web search engine to trace the transformation of a large number of failed searches into questions posted on a popular CQA site. Specifically, we analyze the characteristics of the queries, and of the patterns of search behavior that precede posting a question; the relationship between the content of the attempted queries and of the posted questions; and the subsequent actions the user performs on the CQA site. Our work develops novel insights into searcher intent and behavior that lead to asking questions to the community, providing a foundation for more effective integration of automated web search and social information seeking.
Adaptive query suggestion for difficult queries
Authors: Yang Liu, Ruihua Song, Yu Chen, Jian-Yun Nie, Ji-Rong Wen
Venue: SIGIR
Year: 2012

Query suggestion is a useful tool to help users formulate better queries. Although this has been found highly useful globally, its effect on different queries may vary. In this paper, we examine the impact of query suggestion on queries of different degrees of difficulty. It turns out that query suggestion is much more useful for difficult queries than easy queries. In addition, the suggestions for difficult queries should rely less on their similarity to the original query. In this paper, we use a learning-to-rank approach to select query suggestions, based on several types of features including a query performance prediction. As query suggestion has different impacts on different queries, we propose an adaptive suggestion approach that makes suggestions only for difficult queries. We carry out experiments on real data from a search engine. Our results clearly indicate that an approach targeting difficult queries can bring higher gain than a uniform suggestion approach.
Optimizing parameters of the expected reciprocal rank
Authors: Yury Logachev, Pavel Serdyukov
Venue: SIGIR
Year: 2012

Most popular IR metrics are parameterized. Usually parameters of these metrics are chosen on the basis of general considerations and not adjusted by experiments with real users. Particularly, the parameters of the Expected Reciprocal Rank measure are the normalized parameters of the DCG metric, and the latter are chosen in an ad-hoc manner. We suggest an approach for adjusting parameters of the ERR metric that allows to reach maximum agreement with the real users behavior. More exactly, we optimized the parameters by maximizing Pearson weighted correlation between ERR and several online click metrics. For each click metric we managed to find the parameters of ERR that result into its higher correlation with the given online click metric.
Fixed versus dynamic co-occurrence windows in TextRank term weights for information retrieval
Authors: Wei Lu, Qikai Cheng, Christina Lioma
Venue: SIGIR
Year: 2012

TextRank is a variant of PageRank typically used in graphs that represent documents, and where vertices denote terms and edges denote relations between terms. Quite often the relation between terms is simple term co-occurrence within a fixed window of k terms. The output of TextRank when applied iteratively is a score for each vertex, i.e. a term weight, that can be used for information retrieval (IR) just like conventional term frequency based term weights. So far, when computing TextRank term weights over co-occurrence graphs, the window of term co-occurrence is always fixed. This work departs from this, and considers dynamically adjusted windows of term co-occurrence that follow the document structure on a sentence- and paragraph-level. The resulting TextRank term weights are used in a ranking function that re-ranks 1000 initially returned search results in order to improve the precision of the ranking. Experiments with two IR collections show that adjusting the vicinity of term co-occurrence when computing TextRank term weights can lead to gains in early precision.
Patent information retrieval: an instance of domain-specific search
Authors: Mihai Lupu
Venue: SIGIR
Year: 2012

The tutorial aims to provide the IR researchers with an understanding of how the patent system works, the challenges that patent searchers face in using the existing tools and in adopting new methods developed in academia.
An exploration of ranking heuristics in mobile local search
Authors: Yuanhua Lv, Dimitrios Lymberopoulos, Qiang Wu
Venue: SIGIR
Year: 2012

Users increasingly rely on their mobile devices to search local entities, typically businesses, while on the go. Even though recent work has recognized that the ranking signals in mobile local search (e.g., distance and customer rating score of a business) are quite different from general Web search, they have mostly treated these signals as a black-box to extract very basic features (e.g., raw distance values and rating scores) without going inside the signals to understand how exactly they affect the relevance of a business. However, as it has been demonstrated in the development of general information retrieval models, it is critical to explore the underlying behaviors/heuristics of a ranking signal to design more effective ranking features.
New assessment criteria for query suggestion
Authors: Zhongrui Ma, Yu Chen, Ruihua Song, Tetsuya Sakai, Jiaheng Lu, Ji-Rong Wen
Venue: SIGIR
Year: 2012

Query suggestion is a useful tool to help users express their information needs by supplying alternative queries. When evaluating the effectiveness of query suggestion algorithms, many previous studies focus on measuring whether a suggestion query is relevant or not to the input query. This assessment criterion is too simple to describe users' requirements. In this paper, we introduce two scenarios of query suggestion. The first scenario represents cases where the search result of the input query is unsatisfactory. The second scenario represents cases where the search result is satisfactory but the user may be looking for alternative solutions. Based on the two scenarios, we propose two assessment criteria. Our labeling results indicate that the new assessment criteria provide finer distinctions among query suggestions than the traditional relevance-based criterion.
User-aware caching and prefetching query results in web search engines
Authors: Hongyuan Ma, Bin Wang
Venue: SIGIR
Year: 2012

Query results caching is an efficient technique for Web search engines. In this paper we present User-Aware Cache, a novel approach tailored for query results caching, that is based on user characteristics. We then use a trace of around 30 million queries to evaluate User-Aware Cache, as well as traditional methods and theoretical upper bounds. Experimental results show that this approach can achieve hit ratios better than state-of-the-art methods.
Effect of dynamic pruning safety on learning to rank effectiveness
Authors: Craig Macdonald, Nicola Tonellotto, Iadh Ounis
Venue: SIGIR
Year: 2012

A dynamic pruning strategy, such as WAND, enhances retrieval efficiency without degrading effectiveness to a given rank K, known as safe-to-rank-K. However, it is also possible for WAND to obtain more efficient but unsafe retrieval without actually significantly degrading effectiveness. On the other hand, in a modern search engine setting, dynamic pruning strategies can be used to efficiently obtain the set of documents to be re-ranked by the application of a learned model in a learning to rank setting. No work has examined the impact of safeness on the effectiveness of the learned model. In this work, we investigate the impact of WAND safeness through experiments using 150 TREC Web track topics. We find that unsafe WAND is biased towards documents with lower docids, thereby impacting effectiveness.
Automatic refinement of patent queries using concept importance predictors
Authors: Parvaz Mahdabi, Linda Andersson, Mostafa Keikha, Fabio Crestani
Venue: SIGIR
Year: 2012

Patent prior art queries are full patent applications which are much longer than standard web search topics. Such queries are composed of hundreds of terms and do not represent a focused information need. One way to make the queries more focused is to select a group of key terms as representatives. Existing works show that such a selection to reduce patent queries is a challenging task mainly because of the presence of ambiguous terms. Given this setup, we present a query modeling approach where we utilize patent-specific characteristics to generate more precise queries. We propose to automatically disambiguate query terms by employing noun phrases that are extracted using the global analysis of the patent collection. We further introduce a method for predicting whether expansion using noun phrases would improve the retrieval effectiveness.
myDJ: recommending karaoke songs from one's own voice
Authors: Kuang Mao, Xinyuan Luo, Ke Chen, Gang Chen, Lidan Shou
Venue: SIGIR
Year: 2012

In this demo, we present myDJ, a karaoke recommendation system which recommends the songs people are capable to sing. Different from the existing song recommendation systems which recommend songs people like to listen, myDJ can recommend proper songs according to a subject's physical phonation area. It consists of a singer profiler to analyze the subject's phonation characters. In addition, the song profile for each song in database is extracted. To learn a ranking function, the learning to rank algorithm Listnet is applied under a list of predefined features extracted from each singer-song profile pair. In the results, proper songs which are suitable but challenging for the subject are recommended.
Unsupervised linear score normalization revisited
Authors: Ilya Markov, Avi Arampatzis, Fabio Crestani
Venue: SIGIR
Year: 2012

We give a fresh look into score normalization for merging result-lists, isolating the problem from other components. We focus on three of the simplest, practical, and widely-used linear methods which do not require any training data, i.e. MinMax, Sum, and Z-Score. We provide theoretical arguments on why and when the methods work, and evaluate them experimentally. We find that MinMax is the most robust under many circumstances, and that Sum is - in contrast to previous literature - the worst. Based on the insights gained, we propose another three simple methods which work as good or better than the baselines.
Visual information retrieval using Java and LIRE
Authors: Oge Marques, Mathias Lux
Venue: SIGIR
Year: 2012

Visual information retrieval (VIR) is an active and vibrant research area, which attempts at providing means for organizing, indexing, annotating, and retrieving visual information (images and videos) form large, unstructured repositories. The goal of VIR is to retrieve the highest number of relevant matches to a given query (often expressed as an example image and/or a series of keywords). In its early years (1995-2000) the research efforts were dominated by content-based approaches contributed primarily by the image and video processing community. During the past decade, it was widely recognized that the challenges imposed by the semantic gap (the lack of coincidence between an image's visual contents and its semantic interpretation) required a clever use of textual metadata (in addition to information extracted from the image's pixel contents) to make image and video retrieval solutions efficient and effective. The need to bridge (or at least narrow) the semantic gap has been one of the driving forces behind current VIR research. Additionally, other related research problems and market opportunities have started to emerge, offering a broad range of exciting problems for computer scientists and engineers to work on. In this tutorial, we present an overview of visual information retrieval (VIR) concepts, techniques, algorithms, and applications. Several topics are supported by examples written in Java, using Lucene (an open-source Java-based indexing and search implementation) and LIRE (Lucene Image REtrieval), an open-source Java-based library for content-based image retrieval (CBIR) written by Mathias Lux.
Retrieving information from the book of humanity: the personalized medicine data tsunami crashes on the beach of jeopardy
Authors: Daniel R. Masys
Venue: SIGIR
Year: 2012

From a mute but eloquent alphabet of 4 characters emerges a complex biological 'literature' whose highest expression is human existence. The rapidly advancing technologies of 'nextgen sequencing' will soon make it possible to inexpensively acquire and store the characters of our complete personal genetic instruction set and make it available for health assessment and disease management. This uniquely personal form of 'big data' brings with it challenges that will be discussed in this keynote presentation. Topics will include a brief introduction to the linguistic challenges of 'biology as literature', the impact of personal molecular variation on traditional approaches to disease prevention, diagnosis and treatment, and the challenges of information retrieval when a large volume of primary observations is made that is associated with an evanescent and rapidly changing corpus of scientific interpretation of those primary observations. Experience with extracting high quality pheonotypes from electronic medical records has shown that Natural Language Processing capability is an essential information extraction function for correlation of clinical events with personal genetic variation. Any powerful set of information can be used or misused, and put those who depend upon it in jeopardy. These issues, and a lesson from the long running Jeopardy TV series, will be discussed.
CrowdTerrier: automatic crowdsourced relevance assessments with terrier
Authors: Richard McCreadie, Craig Macdonald, Iadh Ounis
Venue: SIGIR
Year: 2012

In this demo, we present CrowdTerrier, an infrastructure extension to the open source Terrier IR platform that enables the semi-automatic generation of relevance assessments for a variety of document ranking tasks using crowdsourcing. The aim of CrowdTerrier is to reduce the time and expertise required to effectively Crowdsource relevance assessments by abstracting away from the complexities of the crowdsourcing process. It achieves this by automating the assessment process as much as possible, via a close integration of the IR system that ranks the documents (Terrier) and the crowdsourcing marketplace that is used to assess those documents (Amazon's Mechanical Turk).
On building a reusable Twitter corpus
Authors: Richard McCreadie, Ian Soboroff, Jimmy Lin, Craig Macdonald, Iadh Ounis, Dean McCullough
Venue: SIGIR
Year: 2012

The Twitter real-time information network is the subject of research for information retrieval tasks such as real-time search. However, so far, reproducible experimentation on Twitter data has been impeded by restrictions imposed by the Twitter terms of service. In this paper, we detail a new methodology for legally building and distributing Twitter corpora, developed through collaboration between the Text REtrieval Conference (TREC) and Twitter. In particular, we detail how the first publicly available Twitter corpus - referred to as Tweets2011 - was distributed via lists of tweet identifiers and specialist tweet crawling software. Furthermore, we analyse whether this distribution approach remains robust over time, as tweets in the corpus are removed either by users or Twitter itself. Tweets2011 was successfully used by 58 participating groups for the TREC 2011 Microblog track, while our results attest to the robustness of the crawling methodology over time.
Impact of assessor disagreement on ranking performance
Authors: Pavel Metrikov, Virgiliu Pavlu, Javed A. Aslam
Venue: SIGIR
Year: 2012

We consider the impact of inter-assessor disagreement on the maximum performance that a ranker can hope to achieve. We demonstrate that even if a ranker were to achieve perfect performance with respect to a given assessor, when evaluated with respect to a different assessor, the measured performance of the ranker decreases significantly. This decrease in performance may largely account for observed limits on the performance of learning-to-rank algorithms.
Experimental methods for information retrieval
Authors: Donald Metzler, Oren Kurland
Venue: SIGIR
Year: 2012

Experimental evaluation plays a critical role in driving progress in information retrieval (IR) today. There are few alternative ways of exploring, in depth, the empirical merits (or lack thereof) of newly devised search techniques. Careful evaluation is necessary for advancing the state-of-theart; yet, many published papers present work that was illevaluated. Indeed, this phenomenon has garnered attention from the community recently, after the publication of a controversial, but eye-opening, study by Armstrong et al. that suggested ad hoc search quality has not meaningfully advanced since 1984 [1]. The authors noted that the root of the problem was generally lax evaluation methodologies (e.g., weak baselines, etc.). Furthermore, many submissions to top IR research venues (e.g., SIGIR, CIKM, ECIR, WSDM, WWW, etc.) are rejected primarily due to insufficient or inappropriate evaluation. There is therefore a strong need to educate students, researchers, and practitioners about the proper way to carry out IR experiments. This is unfortunately not something that is taught in IR courses or covered in IR textbooks. Indeed, to the best of our knowledge, there is very little written work that lays down some principles for running an IR experiment [3]. More specifically, there have not been any recent tutorials or written works that have specifically, and comprehensively, addressed the question of “how to run an IR experiment” in terms of effectiveness evaluation. This has potentially yielded a number of detrimental effects, as described above. The goal of the tutorial is to provide an initial set of training material for researchers interested in rigorous evaluations of information retrieval systems. Although the primary focus is on ad hoc retrieval experiments, the principles and concepts described in the tutorial are general and can easily be applied to a wide range of experimental scenarios both within, and beyond, the field of information retrieval. The tutorial is primarily inteded for graduate students, re-
Proximity-based rocchio's model for pseudo relevance
Authors: Jun Miao, Jimmy Xiangji Huang, Zheng Ye
Venue: SIGIR
Year: 2012

Rocchio's relevance feedback model is a classic query expansion method and it has been shown to be effective in boosting information retrieval performance. The selection of expansion terms in this method, however, does not take into account the relationship between the candidate terms and the query terms (e.g., term proximity). Intuitively, the proximity between candidate expansion terms and query terms can be exploited in the process of query expansion, since terms closer to query terms are more likely to be related to the query topic. In this paper, we study how to incorporate proximity information into the Rocchio's model, and propose a proximity-based Rocchio's model, called PRoc, with three variants. In our PRoc models, a new concept (proximity-based term frequency, ptf) is introduced to model the proximity information in the pseudo relevant documents, which is then used in three kinds of proximity measures. Experimental results on TREC collections show that our proposed PRoc models are effective and generally superior to the state-of-the-art relevance feedback models with optimal parameters.A direct comparison with positional relevance model (PRM) on the GOV2 collection also indicates our proposed model is at least competitive to the most recent progress.
Multi-platform image search using tag enrichment
Authors: Jinming Min, Cristover Lopes, Johannes Leveling, Dag Schmidtke, Gareth J. F. Jones
Venue: SIGIR
Year: 2012

The number of images available online is growing steadily and current web search engines have indexed more than 10 billion images. Approaches to image retrieval are still often text-based and operate on image annotations and captions. Image annotations (i.e. image tags) are typically short, user-generated, and of varying quality, which increases the mismatch problem between query terms and image tags. For example, a user might enter the query "wedding dress" while all images are annotated with "bridal gown" or "wedding gown". This demonstration presents an image search system using reduction and expansion of image annotations to overcome vocabulary mismatch problems by enriching the sparse set of image tags. Our image search application accepts a written query as input and produces a ranked list of result images and annotations (i.e. image tags) as output. The system integrates methods to reduce and expand the image tag set, thus decreasing the effect of sparse image tags. It builds on different image collections such as the Wikipedia image collection (http://www.imageclef.org/wikidata) and the Microsoft Office.com ClipArt collection (http://office.microsoft.com/), but can be applied to social collections such as Flickr as well. Our demonstration system runs on PCs, tablets, and smartphones, making use of advanced user interface capabilities on mobile devices.