OpenRaiser/Intern-Atlas huggingface dataset:
A repetition based measure for verification of text collections and for text categorization
Authors: Dmitry V. Khmelev, William John Teahan
Venue: SIGIR
Year: 2003

We suggest a way for locating duplicates and plagiarisms in a text collection using an R-measure, which is the normalized sum of the lengths of all suffixes of the text repeated in other documents of the collection. The R-measure can be effectively computed using the suffix array data structure. Additionally, the computation procedure can be improved to locate the sets of duplicate or plagiarised documents. We applied the technique to several standard text collections and found that they contained a significant number of duplicate and plagiarised documents. Another reformulation of the method leads to an algorithm that can be applied to supervised multi-class categorization. We illustrate the approach using the recently available Reuters Corpus Volume 1 (RCV1). The results show that the method outperforms SVM at multi-class categorization, and interestingly, that results correlate strongly with compression-based methods.
Classification of source code archives
Authors: Robert Krovetz, Secil Ugurel, C. Lee Giles
Venue: SIGIR
Year: 2003

The World Wide Web contains a number of source code archives. Programs are usually classified into various categories within the archive by hand. We report on experiments for automatic classification of source code into these categories. We examined a number of factors that affect classification accuracy. Weighting features by expected entropy loss makes a significant improvement in classification accuracy. We show a Support Vector Machine can be trained to classify source code with a high degree of accuracy. We feel these results show promise for software reuse.
Music modeling with random fields
Authors: Victor Lavrenko, Jeremy Pickens
Venue: SIGIR
Year: 2003

this report we discuss an application of Random Fields to the problem of statistical modeling of polyphonic music. With such models in hand, the challenges of developing effective searching, browsing, and organization techniques for the growing bodies of music collections may be successfully met
eArchivarius: accessing collections of electronic mail
Authors: Anton Leuski, Douglas W. Oard, Rahul Bhagat
Venue: SIGIR
Year: 2003

We present eArchivarius an interactive system for accessing collections of electronic mail. The system combines search, clustering visualization, and time-based visualization of email messages and people who send or received the messages.
Assessing the effectiveness of pen-based input queries
Authors: Stephen Levin, Paul D. Clough, Mark Sanderson
Venue: SIGIR
Year: 2003

In this poster, we describe an experiment exploring the effectiveness of a pen based text input device for use in query construction. Standard TREC queries were written, recognised, and subsequently retrieved upon. Comparisons between retrieval effectiveness based on the recognised writing and a typed text baseline were made. On average, effectiveness was 75% of the baseline. Other statistics on the quality and nature of recognition are also reported.
A comparative study on content-based music genre classification
Authors: Tao Li, Mitsunori Ogihara, Qi Li
Venue: SIGIR
Year: 2003

Content-based music genre classification is a fundamental component of music information retrieval systems and has been gaining importance and enjoying a growing amount of attention with the emergence of digital music on the Internet. Currently little work has been done on automatic music genre classification, and in addition, the reported classification accuracies are relatively low. This paper proposes a new feature extraction method for music genre classification, DWCHs. DWCHs stands for Daubechies Wavelet Coefficient Histograms. DWCHs capture the local and global information of music signals simultaneously by computing histograms on their Daubechies wavelet coefficients. Effectiveness of this new feature and of previously studied features are compared using various machine learning classification algorithms, including Support Vector Machines and Linear Discriminant Analysis. It is demonstrated that the use of DWCHs significantly improves the accuracy of music genre classification.
User-trainable video annotation using multimodal cues
Authors: Ching-Yung Lin, Milind R. Naphade, Apostol Natsev, Chalapathy Neti, John R. Smith, Belle L. Tseng, Harriet J. Nock, W. H. Adams
Venue: SIGIR
Year: 2003

This paper describes progress towards a general framework for incorporating multimodal cues into a trainable system for automatically annotating user-defined semantic concepts in broadcast video. Models of arbitrary concepts are constructed by building classifiers in a score space defined by a pre-deployed set of multimodal models. Results show annotation for user-defined concepts both in and outside the pre-deployed set is competitive with our best video-only models on the TREC Video 2002 corpus. An interesting side result shows speech-only models give performance comparable to our best video-only models for detecting visual concepts such as "outdoors", "face" and "cityscape".
A personalised information retrieval tool
Authors: Innes Martin, Joemon M. Jose
Venue: SIGIR
Year: 2003

Industry professionals and everyday users of the Internet have long accepted that due to both the size and growth of this repository, new tools are needed to assist with the finding and extraction of very specific resources relevant to a user&apos;s task. Previously, this definition of relevance has been related to the matching between resources and query terms but recently the emphasis is shifting towards a more personalised model based on the relevance of a particular resource for one specific user. We introduce our system, Fetch, which adopts this concept within an information-seeking environment specifically designed to provide users with means to describe a long-term multifaceted information need. By taking advantage of the way in which users bundle together groups of documents representing a particular topic, query languages as we know them can be taken to a higher and more useful level of abstraction. The agent personalises the search experience by using this information to formulate queries with the aim of returning documents relevant to the user’s information need. In this paper we report on both qualitative and quantitative aspects of system use based on information collected in the pilot evaluation. 1.
Single n-gram stemming
Authors: James Mayfield, Paul McNamee
Venue: SIGIR
Year: 2003

Stemming can improve retrieval accuracy, but stemmers are language-specific. Character n-gram tokenization achieves many of the benefits of stemming in a language independent way, but its use incurs a performance penalty. We demonstrate that selection of a single n-gram as a pseudo-stem for a word can be an effective and efficient language-neutral approach for some languages.
Search strategies in content-based image retrieval
Authors: Sharon McDonald, John Tait
Venue: SIGIR
Year: 2003

This paper describes two studies that looked at users' ability to formulate visual queries with a Content-Based Image Retrieval system that uses dominant image colour as the primary indexing key. The first experiment examined users' performance with two visual search tools, a sketch tool and a structured browsing tool, with different types of image query. The results showed that while users were able to successfully search on the basis of colour, and were able to formulate visual queries, their ability to do so was affected by search task type. Search task type was also shown to be related to search tool choice. However, the results of study two showed that while users were able to complete all of the tasks, there was evidence to suggest that a degree of compromise was present in the users' choice of image that was largely due to problems relating to query formulation.
Statistical visual feature indexes in video retrieval
Authors: Xiangming Mu, Gary Marchionini
Venue: SIGIR
Year: 2003

Four statistical visual feature indexes are proposed: SLM (Shot Length Mean), the average length of each shot in a video; SLD (Shot Length Deviation), the standard deviation of shot lengths for a video; ONM (Object Number Mean), the average number of objects per frame of the video; and OND (Object Number Deviation), the standard deviation of the number of objects per frame across the video. Each of these indexes provides a unique perspective on video content. A novel video retrieval interface has been developed as a platform to examine our assumption that the new indexes facilitate some video retrieval tasks. Initial feedback is promising and formal experiments are planned.
Building and applying a concept hierarchy representation of a user profile
Authors: Nikolaos Nanas, Victoria S. Uren, Anne N. De Roeck
Venue: SIGIR
Year: 2003

Term dependence is a natural consequence of language use. Its successful representation has been a long standing goal for Information Retrieval research. We present a methodology for the construction of a concept hierarchy that takes into account the three basic dimensions of term dependence. We also introduce a document evaluation function that allows the use of the concept hierarchy as a user profile for Information Filtering. Initial experimental results indicate that this is a promising approach for incorporating term dependence in the way documents are filtered.
Evaluating different methods of estimating retrieval quality for resource selection
Authors: Henrik Nottelmann, Norbert Fuhr
Venue: SIGIR
Year: 2003

In a federated digital library system, it is too expensive to query every accessible library. Resource selection is the task to decide to which libraries a query should be routed. Most existing resource selection algorithms compute a library ranking in a heuristic way. In contrast, the decision-theoretic framework (DTF) follows a different approach on a better theoretic foundation: It computes a selection which minimises the overall costs (e.g. retrieval quality, time, money) of the distributed retrieval. For estimating retrieval quality the recall-precision function is proposed. In this paper, we introduce two new methods: The first one computes the empirical distribution of the probabilities of relevance from a small library sample, and assumes it to be representative for the whole library. The second method assumes that the indexing weights follow a normal distribution, leading to a normal distribution for the document scores. Furthermore, we present the first evaluation of DTF by comparing this theoretical approach with the heuristical state-of-the-art system CORI; here we find that DTF outperforms CORI in most cases.
Automatic ranking of retrieval systems in imperfect environments
Authors: Rabia Nuray, Fazli Can
Venue: SIGIR
Year: 2003

The empirical investigation of the effectiveness of information retrieval (IR) systems requires a test collection, a set of query topics, and a set of relevance judgments made by human assessors for each query. Previous experiments show that differences in human relevance assessments do not affect the relative performance of retrieval systems. Based on this observation, we propose and evaluate a new approach to replace the human relevance judgments by an automatic method. Ranking of retrieval systems with our methodology correlates positively and significantly with that of human-based evaluations. In the experiments, we assume a Web-like imperfect environment: the indexing information for all documents is available for ranking, but some documents may not be available for retrieval. Such conditions can be due to document deletions or network problems. Our method of simulating imperfect environments can be used for Web search engine assessment and in estimating the effects of network conditions (e.g., network unreliability) on IR system performance.
Table extraction using conditional random fields
Authors: David Pinto, Andrew McCallum, Xing Wei, W. Bruce Croft
Venue: SIGIR
Year: 2003

The ability to find tables and extract information from them is a necessary component of data mining, question answering, and other information retrieval tasks. Documents often contain tables in order to communicate densely packed, multi-dimensional information. Tables do this by employing layout patterns to efficiently indicate fields and records in two-dimensional form.Their rich combination of formatting and content present difficulties for traditional language modeling techniques, however. This paper presents the use of conditional random fields (CRFs) for table extraction, and compares them with hidden Markov models (HMMs). Unlike HMMs, CRFs support the use of many rich and overlapping layout and language features, and as a result, they perform significantly better. We show experimental results on plain-text government statistical reports in which tables are located with 92% F1, and their constituent lines are classified into 12 table-related categories with 94% accuracy. We also discuss future work on undirected graphical models for segmenting columns, finding cells, and classifying them as data cells or label cells.
Enhancing cross-language information retrieval by an automatic acquisition of bilingual terminology from comparable corpora
Authors: Fatiha Sadat, Masatoshi Yoshikawa, Shunsuke Uemura
Venue: SIGIR
Year: 2003

This paper presents an approach to bilingual lexicon extraction from comparable corpora and evaluations on Cross-Language Information Retrieval. We explore a bi-directional extraction of bilingual terminology primarily from comparable corpora. A combined statistics-based and linguistics-based model to select best translation candidates to phrasal translation is proposed. Evaluations using a large test collection for Japanese-English revealed the proposed combination of bi-directional comparable corpora, bilingual dictionaries and transliteration, augmented with linguistics-based pruning to be highly effective in Cross-Language Information Retrieval.
Exploiting query history for document ranking in interactive information retrieval
Authors: Xuehua Shen, ChengXiang Zhai
Venue: SIGIR
Year: 2003

In this poster,we incorporate user query history, as context information, to improve the retrieval performance in interactive retrieval. Experiments using the TREC data show that incorporating such context information indeed consistently improves the retrieval performance in both average precision and precision at 20 documents.
Building a filtering test collection for TREC 2002
Authors: Ian Soboroff, Stephen E. Robertson
Venue: SIGIR
Year: 2003

Test collections for the filtering track in TREC have typically used either past sets of relevance judgments, or categorized collections such as Reuters Corpus Volume 1 or OHSUMED, because filtering systems need relevance judgments during the experiment for training and adaptation. For TREC 2002, we constructed an entirely new set of search topics for the Reuters Corpus for measuring filtering systems. Our method for building the topics involved multiple iterations of feedback from assessors, and fusion of results from multiple search systems using different search algorithms. We also developed a second set of inexpensive topics based on categories in the document collection. We found that the initial judgments made for the experiment were sufficient; subsequent pooled judging changed system rankings very little. We also found that systems performed very differently on the category topics than on the assessor-built topics.
Incorporating query term dependencies in language models for document retrieval
Authors: Munirathnam Srikanth, Rohini K. Srihari
Venue: SIGIR
Year: 2003

Introduction Recent advances in Information Retrieval are based on using Statistical Language Models (SLM) for representing documents and evaluating their relevance to user queries [6, 3, 4]. Language Modeling (LM) has been explored in many natural language tasks including machine translation and speech recognition [1]. In LM approach to document retrieval, each document, D, is viewed to have its own language model, MD . Given a query, Q, documents are ranked based on the probability, P (Q|MD ), of their language model generating the query. While the LM approach to information retrieval has been motivated from di#erent perspectives [3, 4], most experiments have used smoothed unigram language models that assume term independence for estimating document language models. N-gram, specifically, bigram language models that capture context provided by the previous word(s) perform better than unigram models [7]. Biterm language models [8] that ignore the word order constraint in bigram langu
Word sense disambiguation in information retrieval revisited
Authors: Christopher Stokoe, Michael P. Oakes, John Tait
Venue: SIGIR
Year: 2003

Word sense ambiguity is recognized as having a detrimental effect on the precision of information retrieval systems in general and web search systems in particular, due to the sparse nature of the queries involved. Despite continued research into the application of automated word sense disambiguation, the question remains as to whether less than 90% accurate automated word sense disambiguation can lead to improvements in retrieval effectiveness. In this study we explore the development and subsequent evaluation of a statistical word sense disambiguation system which demonstrates increased precision from a sense based vector space retrieval model over traditional TF*IDF techniques.
Empirical development of an exponential probabilistic model for text retrieval: using textual analysis to build a better model
Authors: Jaime Teevan, David R. Karger
Venue: SIGIR
Year: 2003

Much work in information retrieval focuses on using a model of documents and queries to derive retrieval algorithms. Model based development is a useful alternative to heuristic development because in a model the assumptions are explicit and can be examined and refined independent of the particular retrieval algorithm. We explore the explicit assumptions underlying the naïve framework by performing computational analysis of actual corpora and queries to devise a generative document model that closely matches text. Our thesis is that a model so developed will be more accurate than existing models, and thus more useful in retrieval, as well as other applications. We test this by learning from a corpus the best document model. We find the learned model better predicts the existence of text data and has improved performance on certain IR tasks.
Searchers' criteria For assessing web pages
Authors: Anastasios Tombros, Ian Ruthven, Joemon M. Jose
Venue: SIGIR
Year: 2003

We investigate the criteria used by online searchers when assessing the relevance of web pages to information-seeking tasks. Twenty four searchers were given three tasks each, and indicated the features of web pages which they employed when deciding about the usefulness of the pages. These tasks were presented within the context of a simulated work-task situation. The results of this study provide a set of criteria used by searchers to decide about the utility of web pages. Such criteria have implications for the design of systems that use or recommend web pages, as well as to authors of web pages.
Image classification using hybrid neural networks
Authors: Chih-Fong Tsai, Kenneth McGarry, John Tait
Venue: SIGIR
Year: 2003

Use of semantic content is one of the major issues which needs to be addressed for improving image retrieval effectiveness. We present a new approach to classify images based on the combination of image processing techniques and hybrid neural networks. Multiple keywords are assigned to an image to represent its main contents, i.e. semantic content. Images are divided into a number of regions and colour and texture features are extracted. The first classifier, a self-organising map (SOM) clusters similar images based on the extracted features. Then, regions of the representative images of these clusters were labeled and used to train the second classifier, composed of several support vector machines (SVMs). Initial experiments on the accuracy of keyword assignment for a small vocabulary are reported.
Document-self expansion for text categorization
Authors: Yuen-Hsien Tseng, Da-Wei Juang
Venue: SIGIR
Year: 2003

Approaches to increase training examples to hopefully improve classification effectiveness are proposed in this work. The approaches were verified by use of two Chinese collections classified by two top-performing classifiers.
Probabilistic term variant generator for biomedical terms
Authors: Yoshimasa Tsuruoka, Jun&apos, ichi Tsujii
Venue: SIGIR
Year: 2003

This paper presents an algorithm to generate possible variants for biomedical terms. The algorithm gives each variant its generation probability representing its plausibility, which is potentially useful for query and dictionary expansions. The probabilistic rules for generating variants are automatically learned from raw texts using an existing abbreviation extraction technique. Our method, therefore, requires no linguistic knowledge or labor-intensive natural language resource. We conducted an experiment using 83,142 MEDLINE abstracts for rule induction and 18,930 abstracts for testing. The results indicate that our method will significantly increase the number of retrieved documents for long biomedical terms.
ReCoM: reinforcement clustering of multi-type interrelated data objects
Authors: Jidong Wang, Hua-Jun Zeng, Zheng Chen, Hongjun Lu, Li Tao, Wei-Ying Ma
Venue: SIGIR
Year: 2003

Most existing clustering algorithms cluster highly related data objects such as Web pages and Web users separately. The interrelation among different types of data objects is either not considered, or represented by a static feature space and treated in the same ways as other attributes of the objects. In this paper, we propose a novel clustering approach for clustering multi-type interrelated data objects, ReCoM (Reinforcement Clustering of Multi-type Interrelated data objects). Under this approach, relationships among data objects are used to improve the cluster quality of interrelated data objects through an iterative reinforcement clustering process. At the same time, the link structure derived from relationships of the interrelated data objects is used to differentiate the importance of objects and the learned importance is also used in the clustering process to further improve the clustering results. Experimental results show that the proposed approach not only effectively overcomes the problem of data sparseness caused by the high dimensional relationship space but also significantly improves the clustering accuracy.
Experimental result analysis for a generative probabilistic image retrieval model
Authors: Thijs Westerveld, Arjen P. de Vries
Venue: SIGIR
Year: 2003

The main conclusion from the metrics-based evaluation of video retrieval systems at TREC's video track is that non-interactive image retrieval from general collections using visual information only is not yet feasible. We show how a detailed analysis of retrieval results -- looking beyond mean average precision (MAP) scores on topical relevance -- gives significant insight in the main problems with the visual part of the retrieval model under study. Such an analytical approach proves an important addition to standard evaluation measures.