OpenRaiser/Intern-Atlas huggingface dataset:
Mining employment market via text block detection and adaptive cross-domain information extraction
Authors: Tak-Lam Wong, Wai Lam, Bo Chen
Venue: SIGIR
Year: 2009

We have developed an approach for analyzing online job advertisements in different domains (industries) from different regions worldwide. Our approach is able to extract precise information from the text content supporting useful employment market analysis locally and globally. A major component in our approach is an information extraction framework which is composed of two challenging tasks. The first task is to detect unformatted text blocks automatically based on an unsupervised learning model. Identifying these useful text blocks through this learning model allows the generation of highly effective features for the next task which is text fragment extraction learning. The task of text fragment extraction learning is formulated as a domain adaptation model for text fragment classification. One advantage of our approach is that it can easily adapt to a large number of online job advertisements in different and new domains. Extensive experiments have been conducted to demonstrate the effectiveness and flexibility of our approach.
Evaluating web search using task completion time
Authors: Ya Xu, David Mease
Venue: SIGIR
Year: 2009

We consider experiments to measure the quality of a web search algorithm based on how much total time users take to complete assigned search tasks using that algorithm. We first analyze our data to verify that there is in fact a negative relationship between a user's total search time and a user's satisfaction for the types of tasks under consideration. Secondly, we fit a model with the user's total search time as the response to compare two different search algorithms. Finally, we propose an alternative experimental design which we demonstrate to be a substantial improvement over our current design in terms of variance reduction and efficiency.
Compressing term positions in web indexes
Authors: Hao Yan, Shuai Ding, Torsten Suel
Venue: SIGIR
Year: 2009

Large search engines process thousands of queries per second on billions of pages, making query processing a major factor in their operating costs. This has led to a lot of research on how to improve query throughput, using techniques such as massive parallelism, caching, early termination, and inverted index compression. We focus on techniques for compressing term positions in web search engine indexes. Most previous work has focused on compressing docID and frequency data, or position information in other types of text collections. Compression of term positions in web pages is complicated by the fact that term occurrences tend to cluster within documents but not across document boundaries, making it harder to exploit clustering effects. Also, typical access patterns for position data are different from those for docID and frequency data. We perform a detailed study of a number of existing and new techniques for compressing position data in web indexes. We also study how to efficiently access position data for ranking functions that take proximity features into account.
Feature selection for automatic taxonomy induction
Authors: Hui Yang, Jamie Callan
Venue: SIGIR
Year: 2009

Most existing automatic taxonomy induction systems exploit one or more features to induce a taxonomy; nevertheless there is no systematic study examining which are the best features for the task under various conditions. This paper studies the impact of using different features on taxonomy induction for different types of relations and for terms at different abstraction levels. The evaluation shows that different conditions need different technologies or different combination of the technologies. In particular, co-occurrence and lexico-syntactic patterns are good features for is-a, sibling and part-of relations; contextual, co-occurrence, patterns, and syntactic features work well for concrete terms; co-occurrence works well for abstract terms.
Protein identification as an information retrieval problem
Authors: Yiming Yang, Subramaniam Ganapathy, Abhay Harpale
Venue: SIGIR
Year: 2009

We present the first interdisciplinary work on transforming a popular problem in proteomics, i.e. protein identification from tandem mass spectra, to an Information Retrieval (IR) problem. We present an empirical comparison of popular IR approaches, such as those available from Indri and Lemur toolkits on benchmark datasets, to representative popular baselines in the proteomics literature. Our experiments demonstrate statistically significant evidence that popular IR approaches outperform representative baseline approaches in proteomics.
Personalized music emotion recognition
Authors: Yi-Hsuan Yang, Yu-Ching Lin, Homer H. Chen
Venue: SIGIR
Year: 2009

In recent years, there has been a dramatic proliferation of research on information retrieval based on highly subjective concepts such as emotion, preference and aesthetic. Such retrieval methods are fascinating but challenging since it is difficult to built a general retrieval model that performs equally well to everyone. In this paper, we propose two novel methods, bag-of-users model and residual modeling, to accommodate the individual differences for emotion-based music retrieval. The proposed methods are intuitive and generally applicable to other information retrieval tasks that involve subjective perception. Evaluation result shows the effectiveness of the proposed methods.
Query sampling for ranking learning in web search
Authors: Linjun Yang, Li Wang, Bo Geng, Xian-Sheng Hua
Venue: SIGIR
Year: 2009

Learning to rank has become a popular approach to build a ranking model for Web search recently. Based on our observation, the constitution of the training set will greatly influence the performance of the learned ranking model. Meanwhile, the number of queries in Web search is nearly infinite and the human labeling cost is expensive, hence a subset of queries need to be carefully selected for training. In this paper, we develop a greedy algorithm to sample the queries, by simultaneously taking the query density, difficulty and diversity into consideration. The experimental results on a collected Web search dataset comprising 2024 queries show that the proposed method can lead to a more informative training set for building an effective model.
A graph-based approach to mining multilingual word associations from wikipedia
Authors: Zheng Ye, Xiangji Huang, Hongfei Lin
Venue: SIGIR
Year: 2009

In this paper, we propose a graph-based approach to constructing a multilingual association dictionary from Wikipedia, in which we exploit two kinds of links in Wikipedia articles to associate multilingual words and concepts together in a graph. The mined association dictionary is applied in cross language information retrieval (CLIR) to verify its quality. We evaluate our approach on four CLIR data sets and the experimental results show that it is possible to mine a good multilingual association dictionary from Wikipedia articles.
Searching documentation using text, OCR, and image
Authors: Tom Yeh, Boris Katz
Venue: SIGIR
Year: 2009

We describe a mixed-modality method to index and search software documentation in three ways: plain text, OCR text of embedded figures, and visual features of these figures. Using a corpus of 102 computer books with a total of 62,943 pages and 75,800 figures, we empirically demonstrate that our method achieves better precision/recall than do alternatives based on single modalities.
Deep versus shallow judgments in learning to rank
Authors: Emine Yilmaz, Stephen Robertson
Venue: SIGIR
Year: 2009

Much research in learning to rank has been placed on developing sophisticated learning methods, treating the training set as a given. However, the number of judgments in the training set directly aff ects the quality of the learned system. Given the expense of obtaining relevance judgments for constructing training data, one often has a limited budget in terms of how many judgments he can get. The major problem then is how to distribute this judgment e ffort across diff erent queries. In this paper, we investigate the tradeo ff between the number of queries and the number of judgments per query when training sets are constructed. In particular, we show that up to a limit, training sets with more queries but shallow (less) judgments per query are more cost effective than training sets with less queries but deep (more) judgments per query.
Fast nonparametric matrix factorization for large-scale collaborative filtering
Authors: Kai Yu, Shenghuo Zhu, John D. Lafferty, Yihong Gong
Venue: SIGIR
Year: 2009

With the sheer growth of online user data, it becomes challenging to develop preference learning algorithms that are sufficiently flexible in modeling but also affordable in computation. In this paper we develop nonparametric matrix factorization methods by allowing the latent factors of two low-rank matrix factorization methods, the singular value decomposition (SVD) and probabilistic principal component analysis (pPCA), to be data-driven, with the dimensionality increasing with data size. We show that the formulations of the two nonparametric models are very similar, and their optimizations share similar procedures. Compared to traditional parametric low-rank methods, nonparametric models are appealing for their flexibility in modeling complex data dependencies. However, this modeling advantage comes at a computational price--it is highly challenging to scale them to large-scale problems, hampering their application to applications such as collaborative filtering. In this paper we introduce novel optimization algorithms, which are simple to implement, which allow learning both nonparametric matrix factorization models to be highly efficient on large-scale problems. Our experiments on EachMovie and Netflix, the two largest public benchmarks to date, demonstrate that the nonparametric models make more accurate predictions of user ratings, and are computationally comparable or sometimes even faster in training, in comparison with previous state-of-the-art parametric matrix factorization models.
Improving user confidence in cultural heritage aggregated results
Authors: Junte Zhang, Alia Amin, Henriette S. M. Cramer, Vanessa Evers, Lynda Hardman
Venue: SIGIR
Year: 2009

State of the art web search systems enable aggregation of information from many sources. Users are challenged to assess the reliability of information from different sources. We report on an empirical user study on the effect of displaying credibility ratings of multiple cultural heritage sources (e.g. museum websites, art blogs) on users' search performance and selection. The results of our online interactive study (N=122) show that when explicitly presenting these ratings, people become significantly more confident in their selection of information from aggregated results.
Approximating true relevance distribution from a mixture model based on irrelevance data
Authors: Peng Zhang, Yuexian Hou, Dawei Song
Venue: SIGIR
Year: 2009

Pseudo relevance feedback (PRF), which has been widely applied in IR, aims to derive a distribution from the top n pseudo relevant documents D. However, these documents are often a mixture of relevant and irrelevant documents. As a result, the derived distribution is actually a mixture model, which has long been limiting the performance of PRF. This is particularly the case when we deal with difficult queries where the truly relevant documents in D are very sparse. In this situation, it is often easier to identify a small number of seed irrelevant documents, which can form a seed irrelevant distribution. Then, a fundamental and challenging problem arises: solely based on the mixed distribution and a seed irrelevance distribution, how to automatically generate an optimal approximation of the true relevance distribution? In this paper, we propose a novel distribution separation model (DSM) to tackle this problem. Theoretical justifications of the proposed algorithm are given. Evaluation results from our extensive simulated experiments on several large scale TREC data sets demonstrate the effectiveness of our method, which outperforms a well respected PRF Model, the Relevance Model (RM), as well as the use of RM on D with the seed negative documents directly removed.
What queries are likely to recur in web search?
Authors: Dell Zhang, Jinsong Lu
Venue: SIGIR
Year: 2009

We study the recurrence dynamics of queries in Web search by analysing a large real-world query log dataset. We find that query frequency is more useful in predicting collective query recurrence whereas query recency is more useful in predicting individual query recurrence. Our findings provide valuable insights for understanding and improving Web search.
Modeling search response time
Authors: Dan Zhang, Luo Si
Venue: SIGIR
Year: 2009

Modeling the response time of search engines is an important task for many applications such as resource selection in federated text search. Limited research has been conducted to address this task. Prior research calculated the search response time of all queries in the same way either with the average response time of several sample queries or with a single probability distribution, which is irrelevant to the characteristics of queries. However, the search response time may vary a lot for different types of queries. This paper proposes a novel query-specific and source-specific approach to model search response time. Some training data is acquired by measuring the search response time of some sample queries from a search engine. Then, a query-specific model is estimated with the training data and their corresponding response times by utilizing Ridge Regression. The obtained model can be used to predict search response times for new queries. A set of empirical studies are conducted to show the effectiveness of the proposed method.
Template-independent wrapper for web forums
Authors: Qi Zhang, Yang Shi, Xuanjing Huang, Lide Wu
Venue: SIGIR
Year: 2009

This paper presents a novel work on the task of extracting data from Web forums. Millions of users contribute rich information to Web forum everyday, which has become an important resource for manyWeb applications, such as product opinion retrieval, social network analysis, and so on. The novelty of the proposed algorithm is that it can not only extract the pure text but also distinguish between the original post and replies. Experimental results on a large number of real Web forums indicate that the proposed algorithm can correctly ex
CompositeMap: a novel framework for music similarity measure
Authors: Bingjun Zhang, Jialie Shen, Qiaoliang Xiang, Ye Wang
Venue: SIGIR
Year: 2009

With the continuing advances in data storage and communication technology, there has been an explosive growth of music information from different application domains. As an effective technique for organizing, browsing, and searching large data collections, music information retrieval is attracting more and more attention. How to measure and model the similarity between different music items is one of the most fundamental yet challenging research problems. In this paper, we introduce a novel framework based on a multimodal and adaptive similarity measure for various applications. Distinguished from previous approaches, our system can effectively combine music properties from different aspects into a compact signature via supervised learning. In addition, an incremental Locality Sensitive Hashing algorithm has been developed to support efficient retrieval processes with different kinds of queries. Experimental results based on two large music collections reveal various advantages of the proposed framework including effectiveness, efficiency, adaptiveness, and scalability.
Mining product reviews based on shallow dependency parsing
Authors: Qi Zhang, Yuanbin Wu, Tao Li, Mitsunori Ogihara, Joseph Johnson, Xuanjing Huang
Venue: SIGIR
Year: 2009

This paper presents a novel method for mining product reviews, where it mines reviews by identifying product features, expressions of opinions and relations between them. By taking advantage of the fact that most of product features are phrases, a concept of shallow dependency parsing is introduced, which extends traditional dependency parsing to phrase level. This concept is then implemented for extracting relation between product features and expressions of opinions. Experimental evaluations show that the mining task can benefit from shallow dependency parsing.
Temporal query substitution for ad search
Authors: Wen Zhang, Jun Yan, Shuicheng Yan, Ning Liu, Zheng Chen
Venue: SIGIR
Year: 2009

Recently, information retrieval researchers have witnessed the increasing interest in query substitution for ad search. Most previous works substitute search queries via content based query similarities, and few of them take the temporal characteristics of queries into consideration. In this extended abstract, we propose a novel temporal similarity measurement for query substitution in ad search task. We firstly extract temporal features, such as burst and periodicity, from query frequency curves and then define the temporal query similarity by integrating these new features with the temporal query frequency distribution. Compared to the traditional temporal similarity measurements such as correlation coefficient, our proposed approach is more effective owing to the explicit extraction of high-level semantic query temporal features for similarity measure. The experimental results demonstrate that the proposed similarity measure can make the ads more relevant to user search queries compared to ad search without temporal features.
A proximity language model for information retrieval
Authors: Jinglei Zhao, Yeogirl Yun
Venue: SIGIR
Year: 2009

The proximity of query terms in a document is a very important information to enable ranking models go beyond the "bag of word" assumption in information retrieval. This paper studies the integration of term proximity information into the unigram language modeling. A new proximity language model (PLM) is proposed which views query terms' proximity centrality as the Dirichlet hyper-parameter that weights the parameters of the unigram document language model. Several forms of proximity measure are developed to be used in PLM which could compute a query term's proximate centrality in a specific document. In experiments, the proximity language model is compared with the basic language model and previous works that combine the proximity information with language model using linear score combination. The experiment results show that the proposed model performs better in both top precision and average precision.
Topic (query) selection for IR evaluation
Authors: Jianhan Zhu, Jun Wang, Vishwa Vinay, Ingemar J. Cox
Venue: SIGIR
Year: 2009

The need for evaluating large amounts of topics (queries) makes IR evaluation an uneasy task. In this paper, we study a topic selection problem for IR evaluation. The selection criterion is based on the overall difficulty of the chosen set, as well as the uncertainty of the final IR metric applied to the systems. Our preliminary experiments demonstrate that our approach helps to identify a set of topics that provides confident estimates of systems' performance while keeping the requirement of the query difficulty.
Leveraging sources of collective wisdom on the web for discovering technology synergies
Authors: Cai-Nicolas Ziegler, Stefan Jung
Venue: SIGIR
Year: 2009

One of the central tasks of R&D strategy and portfolio management at large technology companies and research institutions refers to the identification of technological synergies throughout the organization. These efforts are geared towards saving resources by consolidating scattered expertise, sharing best practices, and reusing available technologies across multiple product lines. In the past, this task has been done in a manual evaluation process by technical domain experts. While feasible, the major drawback of this approach is the enormous effort in terms of availability and time: For a structured and complete analysis every combination of any two technologies has to be rated explicitly. We present a novel approach that recommends technological synergies in an automated fashion, making use of abundant collective wisdom from the Web, both in pure textual form as well as classification ontologies. Our method has been deployed for practical support of the synergy evaluation process within our company. We have also conducted empirical evaluations based on randomly selected technology pairs so as to benchmark the accuracy of our approach, as compared to a group of general computer science technologists as well as a control group of domain experts.
Revisiting logical imaging for information retrieval
Authors: Guido Zuccon, Leif Azzopardi, C. J. van Rijsbergen
Venue: SIGIR
Year: 2009

Retrieval with Logical Imaging is derived from belief revision and provides a novel mechanism for estimating the relevance of a document through logical implication (i.e. P(q->d). In this poster, we perform the first comprehensive evaluation of Logical Imaging (LI) in Information Retrieval (IR) across several TREC test Collections. When compared against standard baseline models, we show that LI fails to improve performance. This failure can be attributed to a nuance within the model that means non-relevant documents are promoted in the ranking, while relevant documents are demoted. This is an important contribution because it not only contextualizes the effectiveness of LI, but crucially explains why it fails. By addressing this nuance, future LI models could be significantly improved.
Graphical models for text: a new paradigm for text representation and processing
Authors: Charu C. Aggarwal, Peixiang Zhao
Venue: SIGIR
Year: 2010

Almost all text applications use the well known vector-space model for text representation and analysis. While the vector-space model has proven itself to be an effective and efficient representation for mining purposes, it does not preserve information about the ordering of the words in the representation. In this paper, we will introduce the concept of distance graph representations of text data. Such representations preserve distance and ordering information between the words, and provide a much richer representation of the underlying text. This approach enables knowledge discovery from text which is not possible with the use of a pure vector-space representation, because it loses much less information about the ordering of the underlying words. Furthermore, this representation does not require the development of new mining and management techniques. This is because the technique can also be converted into a structural version of the vector-space representation, which allows the use of all existing tools for text. In addition, existing techniques for graph and XML data can be directly leveraged with this new representation. Thus, a much wider spectrum of algorithms is available for processing this representation.
multi Searcher: can we support people to get information from text they can't read or understand?
Authors: Farag Ahmed, Andreas Nürnberger
Venue: SIGIR
Year: 2010

The goal of the proposed tool multi Searcher is to answer this research question: can we expect people to be able to get information from text in languages they can not read or understand? The proposed tool multi Searcher provides users with interactive contextual information that describes the translation in the user's own language so that the user has a certain degree of confidence about the translation. Therefore, the user is considered as an integral part of the retrieval process. The tool provides possibilities to interactively select relevant terms from contextual information in order to improve the translation and thus improve the cross lingual information retrieval (CLIR) process.