OpenRaiser/Intern-Atlas huggingface dataset:
Experiments in high-dimensional text categorization
Authors: Fred Damerau, Tong Zhang, Sholom M. Weiss, Nitin Indurkhya
Venue: SIGIR
Year: 2002

We present results for automated text categorization of the Reuters-810000 collection of news stories. Our experiments use the entire one-year collection of 810,000 stories and the entire subject index. We divide the data into monthly groups and provide an initial benchmark of text categorization performance on the complete collection. Experimental results show that efficient sparse-feature implementations of linear methods and decision trees, using a global unstemmed dictionary, can readily handle applications of this size. Predictive performance is approximately as strong as the best results for the much smaller older Reuters collections. Detailed results are provided over time periods. It is shown that a smaller time horizon does not diminish predictive quality, implying reduced demands for retraining when sample size is large.
CuTeX: a system for extracting data from text tables
Authors: Hasan Davulcu, Saikat Mukherjee, Arvind Seth, I. V. Ramakrishnan
Venue: SIGIR
Year: 2002

A wealth of information relevant for e-commerce often appears in text form. This includes specification and performance data sheets of products, financial statements, product offerings etc. Typically these types of product and financial data are published in tabular form. The only separators between items in the table are white spaces and line separators. We will refer to such tables as text tables. Due to the lack of structure in such tables, the information present is not readily queriable using traditional database query languages like SQL. One way to make it amenable to standard database querying techniques is to extract the data items in the tables and create a database out of the extracted data. But extraction from text tables poses difficulties due to the irregularity of the data in the column.
Do thumbnail previews help users make better relevance decisions about web search results?
Authors: Susan Dziadosz, Raman Chandrasekar
Venue: SIGIR
Year: 2002

We describe an empirical evaluation of the utility of thumbnail previews in web search results. Results pages were constructed to show text-only summaries, thumbnail previews only, or the combination of text summaries and thumbnail previews. We found that in the combination case, users were able to make more accurate decisions about the potential relevance of results than in either of the other versions, with hardly any increase in speed of processing the page as a whole.
The web retrieval task and its evaluation in the third NTCIR workshop
Authors: Koji Eguchi, Keizo Oyama, Emi Ishida, Kazuko Kuriyama, Noriko Kando
Venue: SIGIR
Year: 2002

This paper gives an overview of the evaluation method used for the Web Retrieval Task in the Third NTCIR Workshop, which is currently in progress. In the Web Retrieval Task, we try to assess the retrieval effectiveness of each Web search engine system using a common data set, and attempt to build a re-usable test collection suitable for evaluating Web search engine systems. With these objectives, we have built 100gigabyte and 10-gigabyte document sets, mainly gathered from the ‘.jp ’ domain. Relevance judgment is performed on the retrieved documents, which are written in Japanese or English.
Topic structure modeling
Authors: David A. Evans, James G. Shanahan, Victor Sheftel
Venue: SIGIR
Year: 2002

In this paper, we present a method based on document probes to quantify and diagnose topic structure, distinguishing topics as monolithic, structured, or diffuse. The method also yields a structure analysis that can be used directly to optimize filter (classifier) creation. Preliminary results illustrate the predictive value of the approach on TREC/Reuters-96 topics.
Statistical cross-language information retrieval using n-best query translations
Authors: Marcello Federico, Nicola Bertoldi
Venue: SIGIR
Year: 2002

This paper presents a novel statistical model for cross-language information retrieval. Given a written query in the source language, documents in the target language are ranked by integrating probabilities computed by two statistical models: a query-translation model, which generates most probable term-by-term translations of the query, and a query-document model, which evaluates the likelihood of each document and translation. Integration of the two scores is performed over the set of N most probable translations of the query. Experimental results with values N=1, 5, 10 are presented on the Italian-English bilingual track data used in the CLEF 2000 and 2001 evaluation campaigns.
How Many Bits are Needed to Store Term Frequencies?
Authors: Martin Franz, J. Scott McCarley
Venue: SIGIR
Year: 2002

Search algorithms in most current text retrieval systems use index data structures extracted from the original text documents. In this paper we focus on reducing the size of the indices by reducing the amount of space dedicated to store term frequencies. In experiments using TREC Ad Hoc [2, 3] corpora and query sets, we show that it is possible to store the term frequency in only two bits without decreasing retrieval performance.
Non-linear reading for a structured web indexation
Authors: Mathias Géry
Venue: SIGIR
Year: 2002

The growth of the Web has posed new challenges for Information Retrieval (IR). Most of the current systems are based on traditional models, which have been developed for atomic and independents documents and are not adapted to the Web. A promising research orientation consists of studying the impact of the Web structure on indexing. The HyperDocument model presented in this article is based on essential aspects of information comprehension: content, composition and linear/non-linear reading.
Video retrieval using an MPEG-7 based inference network
Authors: Andrew Graves, Mounia Lalmas
Venue: SIGIR
Year: 2002

This work proposes a model for video retrieval based upon the inference network model. The document network is constructed using video metadata encoded using MPEG-7 and captures information pertaining to the structural aspects (video breakdown into shots and scenes), conceptual aspects (video, scene and shot content) and contextual aspects (context information about the position of conceptual content within the document). The retrieval process a) exploits the distribution of evidence among the shots to perform ranking of different levels of granularity, b) addresses the idea that evidence may be inherited during evaluation, and c) exploits the contextual information to perform constrained queries.
Cross-document summarization by concept classification
Authors: Hilda Hardy, Nobuyuki Shimizu, Tomek Strzalkowski, Ting Liu, Xinyang Zhang, G. Bowden Wise
Venue: SIGIR
Year: 2002

In this paper we describe a Cross Document Summarizer XDoX designed specifically to summarize large document sets (50-500 documents and more). Such sets of documents are typically obtained from routing or filtering systems run against a continuous stream of data, such as a newswire. XDoX works by identifying the most salient themes within the set (at the granularity level that is regulated by the user) and composing an extraction summary, which reflects these main themes. In the current version, XDoX is not optimized to produce a summary based on a few unrelated documents; indeed, such summaries are best obtained simply by concatenating summaries of individual documents. We show examples of summaries obtained in our tests as well as from our participation in the first Document Understanding Conference (DUC).
Term-specific smoothing for the language modeling approach to information retrieval: the importance of a query term
Authors: Djoerd Hiemstra
Venue: SIGIR
Year: 2002

This paper follows a formal approach to information retrieval based on statistical language models. By introducing some simple reformulations of the basic language modeling approach we introduce the notion of importance of a query term. The importance of a query term is an unknown parameter that explicitly models which of the query terms are generated from the relevant documents (the important terms), and which are not (the unimportant terms). The new language modeling approach is shown to explain a number of practical facts of today's information retrieval systems that are not very well explained by the current state of information retrieval theory, including stop words, mandatory terms, coordination level ranking and retrieval using phrases.
Indexing, searching, and retrieving of recorded live presentations with the AOF (authoring on the fly) search engine
Authors: Wolfgang Hürst
Venue: SIGIR
Year: 2002

The tremendous amount of data resulting from the regular usage of tools for automatic presentation recording demand for elaborate search functionality. A detailed analysis of the according multimedia documents is required to allow search at a very detailed level. Unfortunately, the produced data differs significantly from traditional documents. In this demo, we discuss the problems appearing in the presentation retrieval scenario and introduce aofSE, a search engine to study and illustrate these problems as well as to develop and present according solutions and new approaches for this task.
Title language model for information retrieval
Authors: Rong Jin, Alexander G. Hauptmann, ChengXiang Zhai
Venue: SIGIR
Year: 2002

In this paper, we propose a new language model, namely, a title language model, for information retrieval. Different from the traditional language model used for retrieval, we define the conditional probability P(QID) as the probability of using query Q as the title for document D. We adopted the statistical translation model learned from the title and document pairs in the collection to compute the probability P(QID). To avoid the sparse data problem, we propose two new smoothing methods. In the experiments with four different TREC document collections, the title language model for information retrieval with the new smoothing method outperforms both the traditional language model and the vector space model for IR significantly.
Language model for IR using collection information
Authors: Rong Jin, Luo Si, Alexander G. Hauptmann, James P. Callan
Venue: SIGIR
Year: 2002

Information retrieval using meta data can be traced back to the early age of IR where documents are represented by the controlled vocabulary. In this paper, we explore the usage of meta-data information under the framework of language model. We present a new language model that is able to take advantage of the category information for documents to improve the retrieval accuracy. We compare the new language model with the traditional language model over the TREC4 dataset where the collection information for documents is obtained using the k-means clustering method. The new language model outperforms the traditional language model, which verifies our statement.
Hierarchical approach to term suggestion device
Authors: Hideo Joho, Mark Sanderson, Micheline Beaulieu
Venue: SIGIR
Year: 2002

Our demonstration shows the hierarchy system working on a locally run search engine. Hierarchies are dynamically generated from the retrieved documents, and visualised on the menus. When a user selects a term from the hierarchy, the documents linked to the term are listed, and the term is then added to the initial query to rerun a search. Through the demonstration we illustrate how hierarchical presentation of expansion terms is achieved, and how our approach supports users to articulate their information needs using the hierarchy.
A visualisation tool for topic tracking analysis and development
Authors: Gareth J. F. Jones, Steven M. Gabb
Venue: SIGIR
Year: 2002

Topic Detection and Tracking (TDT) research explores the development of algorithms to detect novel events and track their development over time for online reports. Development of these methods requires careful evaluation and analysis. Traditional reductive methods of evaluation only represent some of the available information of algorithm behaviour. We describe a visualisation tool for topic tracking which makes it easy to analysis and compare the temporal behaviour of tracking algorithms.
Topic difference factor extraction between two document sets and its application to text categorization
Authors: Takahiko Kawatani
Venue: SIGIR
Year: 2002

To improve performance in text categorization, it is important to extract distinctive features for each class. This paper proposes topic difference factor analysis (TDFA) as a method to extract projection axes that reflect topic differences between two document sets. Suppose all sentence vectors that compose each document are projected onto projection axes. TDFA obtains the axes that maximize the ratio between the document sets as to the sum of squared projections by solving a generalized eigenvalue problem. The axes are called topic difference factors (TDF's). By applying TDFA to the document set that belongs to a given class and a set of documents that is misclassified as belonging to that class by an existent classifier, we can obtain features that take large values in the given class but small ones in other classes, as well as features that take large values in other classes but small ones in the given class. A classifier was constructed applying the above features to complement the kNN classifier. As the results, the micro averaged F1 measure for Reuters-21578 improved from 83.69 to 87.27%.
A new method of parameter estimation for multinomial naive bayes text classifiers
Authors: Sang-Bum Kim, Hae-Chang Rim, Heui-Seok Lim
Venue: SIGIR
Year: 2002

Multinomial naive Bayes classifiers have been widely used for the probabilistic text classification. However, their parameter estimation method sometimes generates inappropriate probabilities. In this paper, we propose a topic document model approach for naive Bayes text classification, where their parameters are estimated with an expectation from the training documents. Experiments are conducted on Reuters 21578 and 20 Newsgroup collection, and our proposed approach obtained a significant improvement in performace over the conventional approach.
Is natural language an inconvenience or an opportunity for IR?
Authors: Kimmo Koskenniemi
Venue: SIGIR
Year: 2002

Natural language (NL) has evolved to facilitate human communication. It enables the speaker to make the listener's mind wander among her experiences and mental associations roughly according to the intentions of the speaker. The speaker and the listener usually share experiences and expectations, and they use mostly the same units and rules of a shared NL. Written language functions similarly, but in a less interactive way, with fewer possibilities for feedback.Both the symbols of NL (i.e. words or morphemes), and their arrangements are meaningful. Not with universal and precise meanings, but similar enough among different speakers and accurate enough for the communication mostly to succeed.NLs are mostly very large systems. Hundreds of thousands of words and infinitely many possible utterances. Even inflection alone might produce huge numbers of forms, e.g. more than ten thousand distinct forms out of every Finnish verb entry.NL processing (for IR or any other purpose) must cope with phenomena like (1) inflection and compounding, (2) synonymy, (3) polysemy, (4) ambiguity, (5) anaphora and (6) head-modifier relations among words and phrases.Language technology can neutralize much of the effect of these 'inconveniences' inherent with NL, but what kinds of advantages could NL have?
Study of category score algorithms for k-NN classifier
Authors: Huaizhong Kou, Georges Gardarin
Venue: SIGIR
Year: 2002

We analyzes category score algorithms for k-NN classifier found in the literature, including majority voting algorithm (MVA), simple sum algorithm (SSA). MVA and SSA are two mainly used algorithms to estimate score for candidate categories in k-NN classifier systems. Based on the hypothesis that utilization of internal relation between documents and categories could improve system performance, two new weighting score models: concept-based weighting (CBW) score model and term independence-based weighting (IBW) score model are proposed. Our experimental results confirm our hypothesis and show that in the term of precision average IBW and CBW are better than the other score models, while SSA is higher than MVA. According to macro-average F1 CBW performs best. Rocchio-based algorithm (RBA) always performs worst.
Higher precision for two-word queries
Authors: K. L. Kwok
Venue: SIGIR
Year: 2002

Queries have specific properties, and may need individualized methods and parameters to optimize retrieval. Length is one property. We look at how two-word queries may attain higher precision by re-ranking using word co-occurrence evidence in retrieved documents. Co-occurrence within document context is not sufficient, but window context including sentence context evidence can provide precision improvements at low recall region of 4 to 10% using initial retrieval results, and positively affects pseudo-relevance feedback.
The boomerang effect: retrieving scientific documents via the network of references and citations
Authors: Birger Larsen, Peter Ingwersen
Venue: SIGIR
Year: 2002

The theory of polyrepresentation [2, 3] provides a theoretical background for how to exploit such features. In summary, the theory hypothesises that overlaps between different cognitive representations of both users’ information needs as well as documents can be exploited for reducing the uncertainties inherent in IR, and thereby improve the performance of IR systems. Two or more different cognitive representations pointing at the same documents is regarded as multi-evidence of those documents being relevant, and suggests to apply a principle of ‘intentional redundancy’ [2] with the purpose of reducing the uncertainties by placing emphasis on overlaps between representations. Better results are expected when cognitively unlike representations are used, e.g., the document title (made by the author) vs. intellectually assigned descriptors from indexers.
A logistic regression approach to distributed IR
Authors: Ray R. Larson
Venue: SIGIR
Year: 2002

This poster session examines a probabilistic approach to distributed information retrieval using a Logistic Regression algorithm for estimation of collection relevance. The algorithm is compared to other methods for distributed search using test collections developed for distributed search evaluation.
Building thematic lexical resources by term categorization
Authors: Alberto Lavelli, Bernardo Magnini, Fabrizio Sebastiani
Venue: SIGIR
Year: 2002

We discuss the automatic generation of thematic lexicons by means of term categorization, a novel task employing techniques from information retrieval (IR) and machine learning (ML). Specifically, we view the generation of such lexicons as an iterative process of learning previously unknown associations between terms and themes (i.e. disciplines, or fields of activity). The process is iterative, in that it generates, for each ci in a set C = {c1,...,cm} of themes, a sequence Li0⊆ Li1⊆ ... ⊆ Lin of lexicons, bootstrapping from an initial lexicon Li0 and a set of text corpora Θ = {θ0,...,θn-1} given as input. The method is inspired by text categorization, the discipline concerned with labelling natural language texts with labels from a predefined set of themes, or categories. However, while text categorization deals with documents represented as vectors in a space of terms, term categorization deals (dually) with terms represented as vectors in a space of documents, and labels terms (instead of documents) with themes. As a learning device we adopt boosting, since (a) it has demonstrated state-of-the-art effectiveness in a variety of text categorization applications, and (b) it naturally allows for a form of "data cleaning", thereby making the process of generating a thematic lexicon an iteration of generate-and-test steps.
Cross-lingual relevance models
Authors: Victor Lavrenko, Martin Choquette, W. Bruce Croft
Venue: SIGIR
Year: 2002

We propose a formal model of Cross-Language Information Retrieval that does not rely on either query translation or document translation. Our approach leverages recent advances in language modeling to directly estimate an accurate topic model in the target language, starting with a query in the source language. The model integrates popular techniques of disambiguation and query expansion in a unified formal framework. We describe how the topic model can be estimated with either a parallel corpus or a dictionary. We test the framework by constructing Chinese topic models from English queries and using them in the CLIR task of TREC9. The model achieves performance around 95% of the strong mono-lingual baseline in terms of average precision. In initial precision, our model outperforms the mono-lingual baseline by 20%. The main contribution of this work is the unified formal model which integrates techniques that are essential for effective Cross-Language Retrieval.
Text genre classification with genre-revealing and subject-revealing features
Authors: Yong-Bae Lee, Sung-Hyon Myaeng
Venue: SIGIR
Year: 2002

Subject or prepositional content has been the focus of most classification research. Genre or style, on the other hand, is a different and important property of text, and automatic text genre classification is becoming important for classification and retrieval purposes as well as for some natural language processing research. In this paper, we present a method for automatic genre classification that is based on statistically selected features obtained from both subject-classified and genre classified training data. The experimental results show that the proposed method outperforms a direct application of a statistical learner often used for subject classification. We also observe that the deviation formula and discrimination formula using document frequency ratios also work as expected. We conjecture that this dual feature set approach can be generalized to improve the performance of subject classification as well.
Automatic metadata generation & evaluation
Authors: Elizabeth D. Liddy, Eileen Allen, Sarah Harwell, Susan Corieri, Özgür Yilmazel, Necati Ercan Ozgencil, Anne Diekema, Nancy J. McCracken, Joanne Silverstein, Stuart A. Sutton
Venue: SIGIR
Year: 2002

The poster reports on a project in which we are investigating methods for breaking the human metadata-generation bottleneck that plagues Digital Libraries. The research question is whether metadata elements and values can be automatically generated from the content of educational resources, and correctly assigned to mathematics and science educational materials. Natural Language Processing and Machine Learning techniques were implemented to automatically assign values of the GEMgenerate metadata element set tofor learning resources provided by the Gateway for Education (GEM), a service that offers web access to a wide range of educational materials. In a user study, education professionals evaluated the metadata assigned to learning resources by either automatic tagging or manual assignment. Results show minimal difference in the eyes of the evaluators between automatically generated metadata and manually assigned metadata.