OpenRaiser/Intern-Atlas huggingface dataset:
How do users find things with PubMed?: towards automatic utility evaluation with user simulations
Authors: Jimmy Lin, Mark D. Smucker
Venue: SIGIR
Year: 2008

In the context of document retrieval in the biomedical domain, this paper explores the complex relationship between the quality of initial query results and the overall utility of an interactive retrieval system. We demonstrate that a content-similarity browsing tool can compensate for poor retrieval results, and that the relationship between retrieval performance and overall utility is non-linear. Arguments are advanced with user simulations, which characterize the relevance of documents that a user might encounter with different browsing strategies. With broader implications to IR, this work provides a case study of how user simulations can be exploited as a formative tool for automatic utility evaluation. Simulation-based studies provide researchers with an additional evaluation tool to complement interactive and Cranfield-style experiments.
Exploring and measuring dependency trees for informationretrieval
Authors: Chang Liu
Venue: SIGIR
Year: 2008

Natural language processing techniques are believed to hold a tremendous potential to supplement the purely quantitative methods of text information retrieval. This has led to the emergence of a large number of NLP-based IR research projects over the last few years, even though the empirical evidence to support this has often been inadequate. Most contributions of NLP to IR mainly concentrate on document representation and compound term matching strategies. Researchers have noted that the simple term-based representation of document content such as vector representation is usually inadequate for accurate discrimination. The "bag of words" representation does not invoke linguistic considerations and allow modelling of relationships between subsets of words. However, even though a variety of content indicator such as syntactic phrase have been tried and investigated for representing documents rather than single terms in IR systems, the matching strategy over those representation still cannot go beyond traditional statistical techniques that measure term co-occurrence characteristics and proximity in analyzing text structure.
On the evolution of the yahoo! answers QA community
Authors: Yandong Liu, Eugene Agichtein
Venue: SIGIR
Year: 2008

While question answering communities have been gaining popularity for several years, we wonder if the increased popularity actually improves or degrades the user experience. In addition, automatic QA systems, which utilize different sources such as search engines and social media, are emerging rapidly. QA communities have already created abundant resources of millions of questions and hundreds of millions of answers. The question whether they will continue to serve as an effective source of information for web search and question answering is of vital importance. In this poster, we investigate the temporal evolution of a popular QA community –Yahoo! Answers, with respect to its effectiveness in answering three basic types of questions: factoid, opinion and complex questions. Our experiments show that Yahoo! Answers keeps growing rapidly, while its overall quality as an information source for factoid question-answering degrades. However, instead of answering factoid questions, it might be more effective to answer opinion and complex questions.
BrowseRank: letting web users vote for page importance
Authors: Yuting Liu, Bin Gao, Tie-Yan Liu, Ying Zhang, Zhiming Ma, Shuyuan He, Hang Li
Venue: SIGIR
Year: 2008

This paper proposes a new method for computing page importance, referred to as BrowseRank. The conventional approach to compute page importance is to exploit the link graph of the web and to build a model based on that graph. For instance, PageRank is such an algorithm, which employs a discrete-time Markov process as the model. Unfortunately, the link graph might be incomplete and inaccurate with respect to data for determining page importance, because links can be easily added and deleted by web content creators. In this paper, we propose computing page importance by using a 'user browsing graph' created from user behavior data. In this graph, vertices represent pages and directed edges represent transitions between pages in the users' web browsing history. Furthermore, the lengths of staying time spent on the pages by users are also included. The user browsing graph is more reliable than the link graph for inferring page importance. This paper further proposes using the continuous-time Markov process on the user browsing graph as a model and computing the stationary probability distribution of the process as page importance. An efficient algorithm for this computation has also been devised. In this way, we can leverage hundreds of millions of users' implicit voting on page importance. Experimental results show that BrowseRank indeed outperforms the baseline methods such as PageRank and TrustRank in several tasks.
Optical character recognition errors and their effects on natural language processing
Authors: Daniel P. Lopresti
Venue: SIGIR
Year: 2008

Errors are unavoidable in advanced computer vision applications such as optical character recognition, and the noise induced by these errors presents a serious challenge to down-stream processes that attempt to make use of such data. In this paper, we apply a new paradigm we have proposed for measuring the impact of recognition errors on the stages of a standard text analysis pipeline: sentence boundary detection, tokenization, and part-of-speech tagging. Our methodology formulates error classification as an optimization problem solvable using a hierarchical dynamic programming approach. Errors and their cascading effects are isolated and analyzed as they travel through the pipeline. We present experimental results based on a large collection of scanned pages to study the varying impact depending on the nature of the error and the character(s) involved. The problem of identifying tabular structures that should not be parsed as sentential text is also discussed.
On iterative intelligent medical search
Authors: Gang Luo, Chunqiang Tang
Venue: SIGIR
Year: 2008

Searching for medical information on the Web has become highly popular, but it remains a challenging task because searchers are often uncertain about their exact medical situations and unfamiliar with medical terminology. To address this challenge, we have built an intelligent medical Web search engine called iMed, which uses medical knowledge and an interactive questionnaire to help searchers form queries. This paper focuses on iMed's iterative search advisor, which integrates medical and linguistic knowledge to help searchers improve search results iteratively. Such an iterative process is common for general Web search, and especially crucial for medical Web search, because searchers often miss desired search results due to their limited medical knowledge and the task's inherent difficulty. iMed's iterative search advisor helps the searcher in several ways. First, relevant symptoms and signs are automatically suggested based on the searcher's description of his situation. Second, instead of taking for granted the searcher's answers to the questions, iMed ranks and recommends alternative answers according to their likelihoods of being the correct answers. Third, related MeSH medical phrases are suggested to help the searcher refine his situation description. We demonstrate the effectiveness of iMed's iterative search advisor by evaluating it using real medical case records and USMLE medical exam questions.
Task-aware search personalization
Authors: Julia Luxenburger, Shady Elbassuoni, Gerhard Weikum
Venue: SIGIR
Year: 2008

Search personalization has been pursued in many ways, in order to\nprovide better result rankings and better overall search experience\nto individual users.\nHowever, blindly applying personalization to all user queries, for example,\nby a background model derived from the user's long-term query-and-click\nhistory, is not always appropriate for aiding the user in accomplishing her \nactual task.\nUser interests change over time, a user sometimes works on very different \ncategories of tasks\nwithin a short timespan, and history-based personalization\nmay impede a user's desire of discovering new topics.\nIn this paper we propose a personalization framework that is\nselective in a twofold sense. First, it selectively employs\npersonalization techniques for queries that are expected to benefit from prior \nhistory\ninformation, while refraining from undue actions otherwise.\nSecond, we introduce the notion of tasks representing\ndifferent granularity levels of a user profile, ranging from very\nspecific search goals to broad topics, and base our reasoning selectively\non query-relevant user tasks.\nThese considerations are cast into a statistical language model for tasks, \nqueries, and\ndocuments, supporting both judicious query expansion and result re-ranking.\nThe effectiveness of our method is demonstrated by an empirical user study.
Limits of opinion-finding baseline systems
Authors: Craig Macdonald, Ben He, Iadh Ounis, Ian Soboroff
Venue: SIGIR
Year: 2008

In opinion-finding, the retrieval system is tasked with retrieving not just relevant documents, but which also express an opinion towards the query target entity. Most opinion-finding systems are based on a two-stage approach, where initially the system aims to retrieve relevant documents, which are then re-ranked according to the extent to which they are detected to be of an opinionated nature. In this work, we investigate how the underlying 'baseline' retrieval system performance affects the overall opinion-finding performance. We apply two effective opinion-finding techniques to all the baseline runs submitted to the TREC 2007 Blog track, and draw new insights and conclusions.
Web page retrieval in ubiquitous sensor environments
Authors: Takuya Maekawa, Yutaka Yanagisawa, Yasushi Sakurai, Yasue Kishino, Koji Kamei, Takeshi Okadome
Venue: SIGIR
Year: 2008

This paper proposes new concept of query free web search for daily living. We ordinarily benefit from additional information about our daily activities that we are currently engaged in. When washing a coffee maker, for example, we receive the benefit if we obtain such information as 'cleaning a coffee maker with vinegar removes its stain well.' Our proposed method automatically searches for a web page including such information relates to an activity of daily living when the activity is performed. We assume that wireless sensor nodes are attached to daily objects to detect object use; our method makes a query from the names of objects which are used. Then, the method retrieves a web page relates to the activity of daily living by using the query.
Text collections for FIRE
Authors: Prasenjit Majumder, Mandar Mitra, Dipasree Pal, Ayan Bandyopadhyay, Samaresh Maiti, Sukanya Mitra, Aparajita Sen, Sukomal Pal
Venue: SIGIR
Year: 2008

The aim of the Forum for Information Retrieval Evaluation (FIRE) is to create a Cranfield-like evaluation framework in the spirit of TREC, CLEF and NTCIR, for Indian Language Information Retrieval. For the first year, six Indian languages have been selected: Bengali, Hindi, Marathi, Punjabi, Tamil, and Telugu. This poster describes the tasks as well as the document and topic collections that are to be used at the FIRE workshop.
Uncovering deep user context from blogs
Authors: Robert McArthur
Venue: SIGIR
Year: 2008

People’s utterances are fundamentally different to other documents because they are more immediate and less thought through. While this makes them more natural – noisy and unstructured – it provides an unrivalled opportunity to see “inside ” the author, to collect some context. The data requires analysis methods that have a relationship to human information processing: socio-cognitively motivated semantic systems. Using HAL, a method validated by cognitive science, the text from a large number of blog entries was analysed to uncover changes in entries author’s sense-of-self. Sense-of-self was measured through geometric projection of author’s first-person usage onto key indicators of kin and negative emotion words. An example of non-clinical qualitative evaluation affirmed the utility and promise of the technique: that deep personal context can be uncovered from utterances through the appropriate analysis and inference.
Don't have a stemmer?: be un+concern+ed
Authors: Paul McNamee, Charles K. Nicholas, James Mayfield
Venue: SIGIR
Year: 2008

The choice of indexing terms used to represent documents crucially determines how e ective subsequent retrieval will be. IR systems commonly use rule-based stemmers to normalize surface word forms to combat the problem of not finding documents that contain words related to query terms by inflectional or derivational morphology. But such stemmers are not available in all languages. In this paper we explore the effectiveness of unsupervised morphological segmentation as an alternative to stemming using test sets in thirteen European languages. We find that unsupervised segmentation is significantly better than unnormalized words, in several cases by more than 20%. However, rule-based stemming, if available, is better in low complexity languages. We also compare these methods to the use of character n-grams, finding that on average n-grams yield the best performance.
A general optimization framework for smoothing language models on graph structures
Authors: Qiaozhu Mei, Duo Zhang, ChengXiang Zhai
Venue: SIGIR
Year: 2008

Recent work on language models for information retrieval has shown that smoothing language models is crucial for achieving good retrieval performance. Many different effective smoothing methods have been proposed, which mostly implement various heuristics to exploit corpus structures. In this paper, we propose a general and unified optimization framework for smoothing language models on graph structures. This framework not only provides a unified formulation of the existing smoothing heuristics, but also serves as a road map for systematically exploring smoothing methods for language models. We follow this road map and derive several different instantiations of the framework. Some of the instantiations lead to novel smoothing methods. Empirical results show that all such instantiations are effective with some outperforming the state of the art smoothing methods.
Parsimonious relevance models
Authors: Edgar Meij, Wouter Weerkamp, Krisztian Balog, Maarten de Rijke
Venue: SIGIR
Year: 2008

We describe a method for applying parsimonious language models to re-estimate the term probabilities assigned by relevance models. We apply our method to six topic sets from test collections in five different genres. Our parsimonious relevance models (i) improve retrieval effectiveness in terms of MAP on all collections, (ii) significantly outperform their non-parsimonious counterparts on most measures, and (iii) have a precision enhancing effect, unlike other blind relevance feedback methods.
On multiword entity ranking in peer-to-peer search
Authors: Yuval Merhav, Ophir Frieder
Venue: SIGIR
Year: 2008

Previously [2], we postulated the advantage of using entity extraction to implement a new Peer-to-Peer (P2P) search framework for reducing network traffic and providing a trade off between precision and recall. We now propose an entity ranking method designed for the 'short documents' characteristic of P2P, which significantly improves both precision and recall in 'top results' P2P search. We construct a dynamic entity corpus using n-grams statistics and metadata, study its reliability, and use it to identify correlations between user query terms.
Semi-supervised spam filtering: does it work?
Authors: Mona Mojdeh, Gordon V. Cormack
Venue: SIGIR
Year: 2008

The results of the 2006 ECML/PKDD Discovery Challenge suggest that semi-supervised learning methods work well for spam filtering when the source of available labeled examples differs from those to be classified. We have attempted to reproduce these results using data from the 2005 and 2007 TREC Spam Track, and have found the opposite effect: methods like self-training and transductive support vector machines yield inferior classifiers to those constructed using supervised learning on the labeled data alone. We investigate differences between the ECML/PKDD and TREC data sets and methodologies that may account for the opposite results.
Extending language modeling techniques to models of search and browsing activity in a digital library
Authors: G. Craig Murray
Venue: SIGIR
Year: 2008

Users searching for information in a digital library or on the WWW can be modeled as individuals moving through a semantic space by issuing queries and clicking on hyperlinks. As they go, they emit a stream of interaction data. Most of it is linguistic data. Lots of it is captured in logs. Some of it is used to guess what the user is searching for. But to most information retrieval systems, each user interaction is a stateless point in this space. There is a timeline connecting each of these points, but systems seldom make use of this as sequence data, in part because there is no clear way to systematically characterize the meaningful relations within a sequence of user activity. It is a problem of pragmatics as much as it is of semantics--the fact that a user clicked on a particular link, or added a particular term to their query, has meaning primarily in relation to the preceding actions. A remaining challenge in IR is to extract features of the user interaction data that will give meaning to those relations.
From concepts to implementation and visualization: tools from a team-based approach to ir
Authors: Uma Murthy, Ricardo da Silva Torres, Edward A. Fox, Logambigai Venkatachalam, Seungwon Yang, Marcos André Gonçalves
Venue: SIGIR
Year: 2008

Researchers have been studying and developing teaching materials for information retrieval (IR), such as [3]. Toolkits also have been built that provide hands-on experience to students. For example, IR-Toolbox [4] is an effort to close the gap between the students' understanding of IR concepts and real-life indexing and search systems. Such tools might be good for helping students in non-technical areas such as in the Library and Information Science field to develop their conceptual model of search engines. However, they do not cover emerging topics and skills, such as content-based image retrieval (CBIR) and fusion search. Although there is open source software (such as those in http://www.searchtools.com/tools/tools-opensource.html) that can be used to teach basic and advanced IR topics, they require a student to have high-level technical knowledge and to spend a long time to gain a practical understanding of these topics.
Exploiting proximity feature in bigram language model for information retrieval
Authors: Seung-Hoon Na, Jungi Kim, In-Su Kang, Jong-Hyeok Lee
Venue: SIGIR
Year: 2008

Language modeling approaches have been effectively dealing with the dependency among query terms based on N-gram such as bigram or trigram models. However, bigram language models suffer from adjacency-sparseness problem which means that dependent terms are not always adjacent in documents, but can be far from each other, sometimes with distance of a few sentences in a document. To resolve the adjacency-sparseness problem, this paper proposes a new type of bigram language model by explicitly incorporating the proximity feature between two adjacent terms in a query. Experimental results on three test collections show that the proposed bigram language model significantly improves previous bigram model as well as Tao's approach, the state-of-art method for proximity-based method.
Query preprocessing: improving web search through a Vietnamese word tokenization approach
Authors: Doan Nguyen
Venue: SIGIR
Year: 2008

In this poster paper, we propose a novel approach to improve web search relevancy by tokenizing a Vietnamese query text prior submitting it to a search engine. Evaluations demonstrate its effectiveness and practical value.
Separate and inequal: preserving heterogeneity in topical authority flows
Authors: Lan Nie, Brian D. Davison
Venue: SIGIR
Year: 2008

Web pages, like people, are often known by others in a variety of contexts. When those contexts are sufficiently distinct, a page's importance may be better represented by multiple domains of authority, rather than by one that indiscriminately mixes reputations. In this work we determine domains of authority by examining the contexts in which a page is cited. However, we find that it is not enough to determine separate domains of authority; our model additionally determines the local flow of authority based upon the relative similarity of the source and target authority domains. In this way, we differentiate both incoming and outgoing hyperlinks by topicality and importance rather than treating them indiscriminately. We find that this approach compares favorably to other topical ranking methods on two real-world datasets and produces an approximately 10% improvement in precision and quality of the top ten results over PageRank.
Kleio: a knowledge-enriched information retrieval system for biology
Authors: Chikashi Nobata, Philip Cotter, Naoaki Okazaki, Brian Rea, Yutaka Sasaki, Yoshimasa Tsuruoka, Jun&apos, ichi Tsujii, Sophia Ananiadou
Venue: SIGIR
Year: 2008

Kleio is an advanced information retrieval (IR) system developed at the UK National Centre for Text Mining (NaCTeM)1. The system offers textual and metadata searches across MEDLINE and provides enhanced searching functionality by leveraging terminology management technologies.
One-button search extracts wider interests: an empirical study with video bookmarking search
Authors: Masayuki Okamoto, Masaaki Kikuchi, Tomohiro Yamasaki
Venue: SIGIR
Year: 2008

This poster presents an overview of the characteristics of a one-button information retrieval interface with closed captions from TV watching activities, which is intended to lighten the burden of remembering and entering query terms while watching TV. We investigated this interface with an experimental system named Video Bookmarking Search, which estimates query terms from closed captions with named-entity recognition and sentence labeling techniques. According to an empirical evaluation for 1,138 search queries from 206 bookmarks using seven actual TV shows on city life, travel, health, and cuisine, we found wider queries and search results are acceptable through the query-input-free interface, despite the fact that the number of queries and search results that are directly relevant to the users' original intentions is not high. The main reason is a watching user's interest is wider than what is expressed with query terms.
Fine-grained relevance feedback for XML retrieval
Authors: Hanglin Pan, Ralf Schenkel, Gerhard Weikum
Venue: SIGIR
Year: 2008

This demonstration presents an XML IR system that allows users to give feedback of different granularities and types, using Dempster-Shafer theory of evidence to compute expanded and reweighted queries.
Question classification with semantic tree kernel
Authors: Yan Pan, Yong Tang, Luxian Lin, Yemin Luo
Venue: SIGIR
Year: 2008

Question Classification plays an important role in most Question Answering systems. In this paper, we exploit semantic features in Support Vector Machines (SVMs) for Question Classification. We propose a semantic tree kernel to incorporate semantic similarity information. A diverse set of semantic features is evaluated. Experimental results show that SVMs with semantic features, especially semantic classes, can significantly outperform the state-of-the-art systems.