OpenRaiser/Intern-Atlas huggingface dataset:
SugarCube: quantification of topic propagation in the blogosphere using percolation theory
Authors: Ali Azimi Bolourian, Yashar Moshfeghi, C. J. van Rijsbergen
Venue: SIGIR
Year: 2009

Blogs facilitate online debates and discussions for millions of people around the world. Identifying the most popular and prevailing topics discussed in the Blogosphere is a crucial task. This poster describes our novel approach to the quantification of the level of topic propagation in the Blogosphere. Our model uses graph-theoretic representations of the Blogosphere's link structures that allows it to deduce the `Percolation Threshold', which is then used in the quantification and definition of a global topic. The result of our experiments on a blog collection shows that our model is able to quantify the propagation of topics. Moreover, our model is successful in identifying specific topics that propagate throughout the Blogosphere and classifies them as `Global'.
Toponym ambiguity in geographical information retrieval
Authors: Davide Buscaldi
Venue: SIGIR
Year: 2009

The objectives of this research work is to study the effects of toponym (place name) ambiguity in the Geographical Information Retrieval (GIR) task. Our experience with GIR systems shows that toponym ambiguity may be an important factor in the inability of these systems to take advantage from geographical knowledge. Previous studies over ambiguity and Information Retrieval (IR) suggested that disambiguation may be useful in some specific IR scenario. We suppose that GIR may constitute such a scenario. This preliminary study was carried out over the WordNet based, manually disambiguated collection developed for the CLIR-WSD task, using the GeoCLEF collection of 100 geographically related topics. The employed GIR system was based on the GeoWorSE system that participated in GeoCLEF 2008. The experiments were carried out considering the manual disambiguation and comparing this result with those obtained by randomly disambiguating the document collection and those obtained by using always the most common referent. The obtained results show no significant difference in the overall results, although the work gave an insight into some errors that are produced by toponym ambiguity and how they may affect the results. These preliminary results also suggest that WordNet is not a suitable resource for the planned research.
Quantifying performance and quality gains in distributed web search engines
Authors: Berkant Barla Cambazoglu, Vassilis Plachouras, Ricardo Baeza-Yates
Venue: SIGIR
Year: 2009

Distributed search engines based on geographical partitioning of a central Web index emerge as a feasible solution to the immense growth of the Web, user bases, and query traffic. However, there is still lack of research in quantifying the performance and quality gains that can be achieved by such architectures. In this paper, we develop various cost models to evaluate the performance benefits of a geographically distributed search engine architecture based on partial index replication and query forwarding. Specifically, we focus on possible performance gains due to the distributed nature of query processing and Web crawling processes. We show that any response time gain achieved by distributed query processing can be utilized to improve search relevance as the use of complex but more accurate algorithms can now be enabled for document ranking. We also show that distributed Web crawling leads to better Web coverage and try to see if this improves the search quality. We verify the validity of our claims over large, real-life datasets via simulations.
Web derived pronunciations for spoken term detection
Authors: Dogan Can, Erica Cooper, Arnab Ghoshal, Martin Jansche, Sanjeev Khudanpur, Bhuvana Ramabhadran, Michael Riley, Murat Saraclar, Abhinav Sethy, Morgan Ulinski, Christopher M. White
Venue: SIGIR
Year: 2009

Indexing and retrieval of speech content in various forms such as broadcast news, customer care data and on-line media has gained a lot of interest for a wide range of applications, from customer analytics to on-line media search. For most retrieval applications, the speech content is typically first converted to a lexical or phonetic representation using automatic speech recognition (ASR). The first step in searching through indexes built on these representations is the generation of pronunciations for named entities and foreign language query terms. This paper summarizes the results of the work conducted during the 2008 JHU Summer Workshop by the Multilingual Spoken Term Detection team, on mining the web for pronunciations and analyzing their impact on spoken term detection. We will first present methods to use the vast amount of pronunciation information available on the Web, in the form of IPA and ad-hoc transcriptions. We describe techniques for extracting candidate pronunciations from Web pages and associating them with orthographic words, filtering out poorly extracted pronunciations, normalizing IPA pronunciations to better conform to a common transcription standard, and generating phonemic representations from ad-hoc transcriptions. We then present an analysis of the effectiveness of using these pronunciations to represent Out-Of-Vocabulary (OOV) query terms on the performance of a spoken term detection (STD) system. We will provide comparisons of Web pronunciations against automated techniques for pronunciation generation as well as pronunciations generated by human experts. Our results cover a range of speech indexes based on lattices, confusion networks and one-best transcriptions at both word and word fragments levels.
Characterizing the subjectivity of topics
Authors: Marc-Allen Cartright, Elif Aktolga, Jeffrey Dalton
Venue: SIGIR
Year: 2009

A document or web page in isolation may appear completely reasonable, but may represent a biased perspective on the topic being discussed. Given the topic of a document, we propose new metrics provocativeness and balance that suggest when the topic could be controversial. We explore the use of these metrics to characterize the subjectivity of the topics in the TREC Blog Track.
Enhancing topical ranking with preferences from click-through data
Authors: Yi Chang, Anlei Dong, Ciya Liao, Zhaohui Zheng
Venue: SIGIR
Year: 2009

To overcome the training data insufficiency problem for dedicated model in topical ranking, this paper proposes to utilize click-through data to improve learning. The efficacy of click-through data is explored under the framework of preference learning. The empirical experiment on a commercial search engine shows that, the model trained with the dedicated labeled data combined with skip-next preferences could beat the baseline model and the generic model in NDCG5 for 4.9% and 2.4% respectively.
Exploiting memory cues in personal lifelog retrieval
Authors: Yi Chen
Venue: SIGIR
Year: 2009

In recent years personal lifelogs (PLs) have become an emerging field of research. PLs are collections of digital data taken from an individual's life experiences, gathered from both digital and physical worlds. PLs collections can potentially be collected over periods of years and thus can be very large. In our research group, four researchers have been engaged in collection of individual one-year-long PLs data sets. These collections include logs of computer and mobile phone activities, digital photos, in particular passively captured Microsoft SenseCam images, geo-location (via GPS), surrounding people or objects (via Bluetooth), and various biometric data. The complex data types and heterogeneous structure of this corpus brings great challenges to traditional content based information retrieval (IR). Yet, the rich connections integral to personal experience offer exciting potential opportunities to leverage features from human memory and associated models to support retrieval.
Multiple approaches to analysing query diversity
Authors: Paul D. Clough, Mark Sanderson, Murad Abouammoh, Sergio Navarro, Monica Lestari Paramita
Venue: SIGIR
Year: 2009

In this paper we examine user queries with respect to diversity: providing a mix of results across different interpretations. Using two query log analysis techniques (click entropy and reformulated queries), 14.9 million queries from the Microsoft Live Search log were analysed. We found that a broad range of query types may benefit from diversification. Additionally, although there is a correlation between word ambiguity and the need for diversity, the range of results users may wish to see for an ambiguous query stretches well beyond traditional notions of word sense.
Estimating query performance using class predictions
Authors: Kevyn Collins-Thompson, Paul N. Bennett
Venue: SIGIR
Year: 2009

We investigate using topic prediction data, as a summary of document content, to compute measures of search result quality. Unlike existing quality measures such as query clarity that require the entire content of the top-ranked results, class-based statistics can be computed efficiently online, because class information is compact enough to precompute and store in the index. In an empirical study we compare the performance of class-based statistics to their language-model counterparts for predicting two measures: query difficulty and expansion risk. Our findings suggest that using class predictions can offer comparable performance to full language models while reducing computation overhead.
On the relative age of spam and ham training samples for email filtering
Authors: Gordon V. Cormack, José-Marcio Martins da Cruz
Venue: SIGIR
Year: 2009

Email spam filters are commonly trained on a sample of spam and ham (non-spam) messages. We investigate the effect on filter performance of using samples of spam and ham messages sent months before those to be filtered. Our results show that filter performance deteriorates with the overall age of spam and ham samples, but at different rates. Spam and ham samples of different ages may be mixed to advantage, provided temporal cues are elided
Reciprocal rank fusion outperforms condorcet and individual rank learning methods
Authors: Gordon V. Cormack, Charles L. A. Clarke, Stefan Büttcher
Venue: SIGIR
Year: 2009

Reciprocal Rank Fusion (RRF), a simple method for combining the document rankings from multiple IR systems, consistently yields better results than any individual system, and better results than the standard method Condorcet Fuse. This result is demonstrated by using RRF to combine the results of several TREC experiments, and to build a meta-learner that ranks the LETOR 3 dataset better than any previously reported method
Spam filter evaluation with imprecise ground truth
Authors: Gordon V. Cormack, Aleksander Kolcz
Venue: SIGIR
Year: 2009

When trained and evaluated on accurately labeled datasets, online email spam filters are remarkably effective, achieving error rates an order of magnitude better than classifiers in similar applications. But labels acquired from user feedback or third-party adjudication exhibit higher error rates than the best filters -- even filters trained using the same source of labels. It is appropriate to use naturally occuring labels -- including errors -- as training data in evaluating spam filters. Erroneous labels are problematic, however, when used as ground truth to measure filter effectiveness. Any measurement of the filter's error rate will be augmented and perhaps masked by the label error rate. Using two natural sources of labels, we demonstrate automatic and semi-automatic methods that reduce the influence of labeling errors on evaluation, yielding substantially more precise measurements of true filter error rates.
Measuring constraint violations in information retrieval
Authors: Ronan Cummins, Colm O&apos, Riordan
Venue: SIGIR
Year: 2009

Recently, an inductive approach to modelling term-weighting function correctness has provided a number of axioms (constraints), to which all good term-weighting functions should adhere. These constraints have been shown to be theoretically and empirically sound in a number of works. It has been shown that when a term-weighting function breaks one or more of the constraints, it typically indicates sub-optimality of that function. This elegant inductive approach may more accurately model the human process of determining the relevance a document. It is intuitive that a person's notion of relevance changes as terms that are either on or off-topic are encountered in a given document. Ultimately, it would be desirable to be able to mathematically determine the performance of term-weighting functions without the need for test collections.
Entropy-biased models for query representation on the click graph
Authors: Hongbo Deng, Irwin King, Michael R. Lyu
Venue: SIGIR
Year: 2009

Query log analysis has received substantial attention in recent years, in which the click graph is an important technique for describing the relationship between queries and URLs. State-of-the-art approaches based on the raw click frequencies for modeling the click graph, however, are not noise-eliminated. Nor do they handle heterogeneous query-URL pairs well. In this paper, we investigate and develop a novel entropy-biased framework for modeling click graphs. The intuition behind this model is that various query-URL pairs should be treated differently, i.e., common clicks on less frequent but more specific URLs are of greater value than common clicks on frequent and general URLs. Based on this intuition, we utilize the entropy information of the URLs and introduce a new concept, namely the inverse query frequency (IQF), to weigh the importance (discriminative ability) of a click on a certain URL. The IQF weighting scheme is never explicitly explored or statistically examined for any bipartite graphs in the information retrieval literature. We not only formally define and quantify this scheme, but also incorporate it with the click frequency and user frequency information on the click graph for an effective query representation. To illustrate our methodology, we conduct experiments with the AOL query log data for query similarity analysis and query suggestion tasks. Experimental results demonstrate that considerable improvements in performance are obtained with our entropy-biased models. Moreover, our method can also be applied to other bipartite graphs.
Adaptation of offline vertical selection predictions in the presence of user feedback
Authors: Fernando Diaz, Jaime Arguello
Venue: SIGIR
Year: 2009

Web search results often integrate content from specialized corpora known as verticals. Given a query, one important aspect of aggregated search is the selection of relevant verticals from a set of candidate verticals. One drawback to previous approaches to vertical selection is that methods have not explicitly modeled user feedback. However, production search systems often record a variety of feedback information. In this paper, we present algorithms for vertical selection which adapt to user feedback. We evaluate algorithms using a novel simulator which models performance of a vertical selector situated in realistic query traffic.
Users' stopping behaviors and estimates of recall
Authors: Maureen Dostert, Diane Kelly
Venue: SIGIR
Year: 2009

This paper investigates subjects' stopping behaviors and estimates of recall in an interactive information retrieval (IIR) experiment. Subjects completed four recall-oriented search tasks and were asked to estimate how many of the relevant documents they believed they had found after each task. Subjects also responded to an interview question probing their reasons for stopping a search. Results show that most subjects believed they found about 51-60% of the relevant documents and that this estimate was correlated positively with number of documents saved and actual recall, even though subjects' recall estimates were inaccurate. Reasons given for stopping search are also explored.
The impact of crawl policy on web search effectiveness
Authors: Dennis Fetterly, Nick Craswell, Vishwa Vinay
Venue: SIGIR
Year: 2009

Crawl selection policy has a direct influence on Web search effectiveness, because a useful page that is not selected for crawling will also be absent from search results. Yet there has been little or no work on measuring this effect. We introduce an evaluation framework, based on relevance judgments pooled from multiple search engines, measuring the maximum potential NDCG that is achievable using a particular crawl. This allows us to evaluate different crawl policies and investigate important scenarios like selection stability over multiple iterations. We conduct two sets of crawling experiments at the scale of 1~billion and 100~million pages respectively. These show that crawl selection based on PageRank, indegree and trans-domain indegree all allow better retrieval effectiveness than a simple breadth-first crawl of the same size. PageRank is the most reliable and effective method. Trans-domain indegree can outperform PageRank, but over multiple crawl iterations it is less effective and more unstable. Finally we experiment with combinations of crawl selection methods and per-domain page limits, which yield crawls with greater potential NDCG than PageRank.
Selecting hierarchical clustering cut points for web person-name disambiguation
Authors: Jun Gong, Douglas W. Oard
Venue: SIGIR
Year: 2009

Hierarchical clustering is often used to cluster person-names referring to the same entities. Since the correct number of clusters for a given person-name is not known a priori, some way of deciding where to cut the resulting dendrogram to balance risks of over- or under-clustering is needed. This paper reports on experiments in which outcome-specific and result-set measures are used to learn a global similarity threshold. Results on the Web People Search (WePS)-2 task indicate that approximately 85% of the optimal F1 measure can be achieved on held-out data.
Beyond session segmentation: predicting changes in search intent with client-side user interactions
Authors: Qi Guo, Eugene Agichtein
Venue: SIGIR
Year: 2009

Effective search session segmentation "grouping queries according to common task or intent" can be useful for improving relevance, search evaluation, and query suggestion. Previous work has largely attempted to segment search sessions off-line, after the fact. In contrast, we present preliminary investigation of predicting, in real time, whether a user is about to switch interest - that is, whether the user is about to finish the current search and switch to another search task (or stop searching altogether). We explore an approach for this task using client-side user behavior such as clicks, scrolls, and mouse movements, contextualized by the content of the search result pages and previous searches. Our experiments over thousands of real searches show that we can identify context and user behavior patterns that indicate that a user is about to switch to a new search task. These preliminary results can be helpful for more effective query suggestion and personalization.
AdOn: an intelligent overlay video advertising system
Authors: Jinlian Guo, Tao Mei, Falin Liu, Xian-Sheng Hua
Venue: SIGIR
Year: 2009

This paper presents a new video advertising system, called AdOn, which supports intelligent overlay video ads. Unlike most current ad-networks such as Youtube that overlay the ads at fixed positions in the videos (e.g., on the bottom fifth of videos 15 seconds in), AdOn is able to automatically detect a set of spatio-temporal nonintrusive positions and associate the contextually relevant ads with these positions. The overlay positions are obtained on the basis of video structuring, face and text detection, as well as visual saliency analysis, so that the intrusiveness to the users can be minimized. The ads are selected according to content-based multimodal relevance so that advertising relevance can be maximized. AdOn represents one of the first attempts towards intelligent overlay video advertising by leveraging video content analysis techniques.
Named entity recognition in query
Authors: Jiafeng Guo, Gu Xu, Xueqi Cheng, Hang Li
Venue: SIGIR
Year: 2009

This paper addresses the problem of Named Entity Recognition in Query (NERQ), which involves detection of the named entity in a given query and classification of the named entity into predefined classes. NERQ is potentially useful in many applications in web search. The paper proposes taking a probabilistic approach to the task using query log data and Latent Dirichlet Allocation. We consider contexts of a named entity (i.e., the remainders of the named entity in queries) as words of a document, and classes of the named entity as topics. The topic model is constructed by a novel and general learning method referred to as WS-LDA (Weakly Supervised Latent Dirichlet Allocation), which employs weakly supervised learning (rather than unsupervised learning) using partially labeled seed entities. Experimental results show that the proposed method based on WS-LDA can accurately perform NERQ, and outperform the baseline methods.
A latent topic model for linked documents
Authors: Zhen Guo, Shenghuo Zhu, Yun Chi, Zhongfei Zhang, Yihong Gong
Venue: SIGIR
Year: 2009

Documents in many corpora, such as digital libraries and webpages, contain both content and link information. To explicitly consider the document relations represented by links, in this paper we propose a citation-topic (CT) model which assumes a probabilistic generative process for corpora. In the CT model a given document is modeled as a mixture of a set of topic distributions, each of which is borrowed (cited) from a document that is related to the given document. Moreover, the CT model contains a random process for selecting the related documents according to the structure of the generative model determined by links and therefore, the transitivity of the relations among documents is captured. We apply the CT model on the document clustering task and the experimental comparisons against several state-of-the-art approaches demonstrate very promising performances.
A relevance-based topic model for news event tracking
Authors: Viet Ha-Thuc, Yelena Mejova, Christopher G. Harris, Padmini Srinivasan
Venue: SIGIR
Year: 2009

Event tracking is the task of discovering temporal patterns of popular events from text streams. Existing approaches for event tracking have two limitations: scalability and inability to rule out non-relevant portions in text streams. In this study, we propose a novel approach to tackle these limitations. To demonstrate the approach, we track news events across a collection of weblogs spanning a two-month time period.
A robust retrieval system of polyphonic music based on chord progression similarity
Authors: Pierre Hanna, Thomas Rocher, Matthias Robine
Venue: SIGIR
Year: 2009

Retrieval systems for polyphonic music rely on the automatic estimation of similarity between two musical pieces. In the case of symbolic music, existing systems either consider a monophonic reduction based on melody or propose algorithms with high complexity. In this paper, we propose a new approach. Musical pieces are represented as a sequence of chords which are estimated from groups of notes sounding at the same time. A root and a mode are associated to each chord. Local alignment is then applied for estimating a similarity score between these sequences. Experiments performed on MIDI files collected on the Internet show that the system proposed allows the retrieval of different versions of the same song.