Novelty detection based on sentence level patterns
The detection of new information in a document stream is an important component of many potential applications. In this paper, a new novelty detection approach based on the identification of sentence level patterns is proposed. Given a user's information need, some patterns in sentences such as combinations of query words, named entities and phrases, may contain more important and relevant information than single words. Therefore, the proposed novelty detection approach focuses on the identification of previously unseen query-related patterns in sentences. Specifically, a query is preprocessed and represented with patterns that include both query words and required answer types. These patterns are used to retrieve sentences, which are then determined to be novel if it is likely that a new answer is present. An analysis of patterns in sentences was performed with data from the TREC 2002 novelty track and experiments on novelty detection were carried out on data from the TREC 2003 and 2004 novelty tracks. The experimental results show that the proposed pattern-based approach significantly outperforms all three baselines in terms of precision at top ranks.

title: Boosting relevance model performance with query term dependence

title: Optimization strategies for complex queries
abstract: Previous research into the efficiency of text retrieval systems has dealt primarily with methods that consider inverted lists in sequence; these methods are known as term-at-a-time methods. However, the literature for optimizing document-at-a-time systems remains sparse.We present an improvement to the max_score optimization, which is the most efficient known document-at-a-time scoring method. Like max_score, our technique, called term bounded max_score, is guaranteed to return exactly the same scores and documents as an unoptimized evaluation, which is particularly useful for query model research. We simulated our technique to explore the problem space, then implemented it in Indri, our large scale language modeling search engine. Tests with the GOV2 corpus on title queries show our method to be 23% faster than max_score alone, and 61% faster than our document-at-a-time baseline. Our optimized query times are competitive with conventional term-at-a-time systems on this year's TREC Terabyte task.

title: Formal multiple-bernoulli models for language modeling
abstract: Statistical language modeling allows formal methods to be applied to information retrieval. As a result, such methods are preferred over their heuristic tf.idf -based counterparts. In language modeling, a statistical model is estimated for each document in the corpus. Documents are then scored by the likelihood the query was generated by the document’s model. Typically, the underlying model is assumed to be of a specific parametric form. In the past, a number of different assumptions have been made about this distribution. In , documents were modeled by a multiple-Bernoulli distribution. However, the estimation and smoothing techniques used to estimate the model were non-standard and somewhat heuristic. The predominant modeling assumption used today, as described in , is to model documents by a multinomial distribution. Such models may be smoothed in a number of ways . Among these is Bayesian (Dirichlet) smoothing that takes a formal, Bayesian approach to smoothing by assuming a Dirichlet prior over the document model. Unlike Ponte and Croft’s multiple-Bernoulli estimation techniques, the multinomial assumption combined with Bayesian smoothing results in a completely formal statistical model. In this paper, we revisit the multiple-Bernoulli assumption and formalize it by taking a Bayesian approach to estimating smoothed document models.

title: Precision prediction based on ranked list coherence
abstract: We introduce a statistical measure of the coherence of a list of documents called the clarity score. Starting with a document list ranked by the query-likelihood retrieval model, we demonstrate the score's relationship to query ambiguity with respect to the collection. We also show that the clarity score is correlated with the average precision of a query and lay the groundwork for useful predictions by discussing a method of setting decision thresholds automatically. We then show that passage-based clarity scores correlate with average-precision measures of ranked lists of passages, where a passage is judged relevant if it contains correct answer text, which extends the basic method to passage-based systems. Next, we introduce variants of document-based clarity scores to improve the robustness, applicability, and predictive ability of clarity scores. In particular, we introduce the ranked list clarity score that can be computed with only a ranked list of documents, and the weighted clarity score where query terms contribute more than other terms. Finally, we show an approach to predicting queries that perform poorly on query expansion that uses techniques expanding on the ideas presented earlier.

title: Applications of multilingual text retrieval
abstract: The recent enormous increase in the use of networked information access and on-line databases has led to more databases being available in languages other than English. The Center for Intelligent Information Retrieval (CIIR) at the University of Massachusetts is involved in a variety of industrial government, and digital library applications which have a need for multilingual text retrieval. Most information retrieval research, however has been evaluated using English databases and queries, and relatively little is and own about how well advanced statistical techniques that incorporate ranking and term weight perform in different languages. We describe our experience with a range of projects involving text retrieval in Spanish, Japanese and Chinese. The issues covered by these projects include document representation techniques such as morphology and segmentation, query formulation and expansion techniques, relevance feedback and comparisons of retrieval effectiveness with English databases. The results indicate that advanced statistical techniques are effective in a wide range of languages, and that new languages can be incorporated with only moderate effort.

title: Ernest Davis, Representations of Commonsense Knowledge