waashk/dblp huggingface dataset:
goal-oriented methods and meta methods for document classification and their parameter tuning. automatic text classification methods come with various calibration parameters such as thresholds for probabilities in bayesian classifiers or for hyperplane distances in svm classifiers. in a given application context these parameters should be set so as to meet the relative importance of various result quality metrics such as precision versus recall. in this paper we consider classifiers that can accept a document for a topic, reject it, or abstain. we aim to meet the application's goals in terms of accuracy (i.e., avoid false acceptances or rejections) and loss (i.e., limit the fraction of documents for which no decision is made). to this end we investigate restrictive forms of support vector machine classifiers and we develop meta methods that split the training data into subsets for independently trained classifiers and then combine the results of these classifiers. these techniques tend to improve accuracy at the expense of document loss. we develop estimators that help to predict the accuracy and loss for a given setting of the methods' tuning parameters, and a methodology for efficiently deriving a setting that meets the application's goals. our experiments confirm the practical viability of the approach.
a visual interface technique for exploring olap data with coordinated dimension hierarchies. multi-dimensional data occurs in many domains while a wide variety of text based and visual interfaces for querying such data exists. but many of these interfaces are not applicable to olap, as they do not support use of dimension hierarchies for selection and aggregation. we introduce an interface technique which supports visual querying of olap data, that has been implemented in the sgviewer tool. it is based on a data graph rather than a data cube representation of the data. our interface presents each dimension hierarchy in a zoomable panel which supports selection and aggregation at multiple levels. users explore data and query it by making selections in several dimension views. three view coordinations are identified; progressive, global and result only. our main contribution, the progressive view coordination provides better support for query refinement than existing interfaces, by helping users decide the next query step with intermediate result overviews, and by helping users change a previous selection decision with retained selection context views. our interface technique is demonstrated with a web log dataset of visits organised into time, download, visitor and referrer address dimensions.
processing content-oriented xpath queries. document-centric xml collections contain text-rich documents, marked up with xml tags that add lightweight semantics to the text. querying such collections calls for a hybrid query language: the text-rich nature of the documents suggests a content-oriented (ir) approach, while the mark-up allows users to add structural constraints to their ir queries. hybrid queries tend to be more expressive, which should lead---in principle---to better retrieval performance. in practice, the processing of these hybrid queries within an ir systems turns out to be far from trivial, because a delicate balance between structural and content information needs to be sought. we propose an approach to processing such hybrid content-and-structure queries that decomposes a query into multiple content-only queries whose results are then combined in ways determined by the structural constraints of the original query. we evaluate our methods using the inex 2003 test-suite, and show (1) that effective ways of processing of content-oriented xpath queries are non-trivial, (2) that there are differences in the effectiveness for different topics types, but (3) that with appropriate processing methods retrieval effectiveness can improve.
viper: augmenting automatic information extraction with visual perceptions. in this paper we address the problem of unsupervised web data extraction. we show that unsupervised web data extraction becomes feasible when supposing pages that are made up of repetitive patterns, as it is the case, e.g., for search engine result pages. hereby the extraction rules are generated automatically without any training or human interaction, by means of operating on the dom tree respectively the flat tag token sequence of a single page.our contribution to automatic data extraction through this paper is twofold. first, we identify and rank potential repetitive patterns with respect to the user's visual perception of the web page, well aware that location and size of matching elements within a web page constitute important criteria for defining relevance. second, matching sub-sequences of the pattern with the highest weightiness are aligned with global multiple sequence alignment techniques. experimental results show that our system is able to achieve high accuracy in distilling and aligning regularly structured objects inside complex web pages.
high dimensional reverse nearest neighbor queries. reverse nearest neighbor (rnn) queries are of particular interest in a wide range of applications such as decision support systems, profile based marketing, data streaming, document databases, and bioinformatics. the earlier approaches to solve this problem mostly deal with two dimensional data. however most of the above applications inherently involve high dimensions and high dimensional rnn problem is still unexplored. in this paper, we propose an approximate solution to answer rnn queries in high dimensions. our approach is based on the strong correlation in practice between k-nn and rnn. it works in two phases. in the first phase the k-nn of a query point is found and in the next phase they are further analyzed using a novel type of query boolean range query (brq). experimental results show that brq is much more efficient than both nn and range queries, and can be effectively used to answer rnn queries. performance is further improved by running multiple brq simultaneously. the proposed approach can also be used to answer other variants of rnn queries such as rnn of order k, bichromatic rnn, and matching query which has many applications of its own. our technique can efficiently answer nn, rnn, and its variants with approximately same number of i/o as running a nn query.
automated cleansing for spend analytics. the development of an aggregate view of the procurement spend across an enterprise using transactional data is increasingly becoming a very important and strategic activity. not only does it provide a complete and accurate picture of what the enterprise is buying and from whom, it also allows it to consolidate suppliers, as well as negotiate better prices. the importance, as well as the complexity, of this cleansing exercise is further magnified by the increasing popularity of business transformation outsourcing (bto) wherein enterprises are turning over non-core activities, such as indirect procurement, to third parties, who now need to develop an integrated view of spend across multiple enterprises in order to optimize procurement and generate maximum savings. however, the creation of such an integrated view of procurement spend requires the creation of a homogeneous data repository from disparate (heterogeneous) data sources across various geographic and functional organizations throughout the enterprise(s). such repositories get transactional data from various sources such as invoices, purchase orders, account ledgers. as such, the transactions are not cross-indexed, refer to the same suppliers by different names, and use different ways of representing information about the same commodities. before an aggregated spend view can be developed, this data needs to be cleansed, primarily to normalize the supplier names and correctly map each transaction to the appropriate commodity code. commodity mapping, in particular, is made more difficult by the fact that it has to be done on the basis of unstructured text descriptions found in the various data sources. we describe an on-demand system to automatically perform this cleansing activity using techniques from information retrieval and machine learning. built on standard integration and application infrastructure software, this system provides enterprises with a fast, reliable, accurate and on-demand way of cleansing transactional data and generating an integrated view of spend. this system is currently in the process of being deployed by ibm for use in its bto practice.
design of a data warehouse system for network/web services. this paper describes the architecture and design of a data warehouse for at&t business services. the main purpose of our system is to generate reports about the performance and reliability of the network. we describe the architecture of our system and discuss some open research problems in this area.
similarity based retrieval from sequence databases using automata as queries. similarity based retrieval from sequence databases is of importance in many applications such as time-series, video and textual databases. in this paper, automata based formalisms are introduced for specifying queries over such databases. various measures defining the distance of a database sequence from an automaton are defined. efficient methods for similarity based retrieval are presented for each of the distance measures. these methods answer nearest neighbor queries (i.e. retrieval of k closest subsequences), or range queries (i.e., retrieval of all sequences with in a given distance).
an adaptive view element framework for multi-dimensional data management. we present an adaptive wavelet view element framework for managing different types of multi-dimensional data in storage and retrieval applications. we consider the problems of multi-dimensional data compression, multi-resolution subregion access, selective materialization, progressive retrieval and similarity searching. the framework uses wavelets to partition the multi-dimensional data into view elements that form the building blocks for synthesizing views of the data. the view elements are organized and managed using different view element graphs. the graphs are used to guide cost-based view element selection algorithms for optimizing compression, access, retrieval and search performance.we present the adaptive wavelet view element framework and describe its application in managing multi-dimensional data such as 1-d time series data, 2-d images, video sequences, and multi-dimensional data cubes. we present experimental results that demonstrate that the adaptive wavelet view element framework improves performance of compressing, accessing, and retrieving multi-dimensional data compared to non-adaptive methods.
securely sharing neuroimagery. shared scientific data, such as neuroimagery, offers great benefits to science. however, data owners must exercise custodial responsibilities which can conflict with the unhindered sharing of their data. given simple choices of sharing widely or not at all, the result will frequently be no sharing. we hypothesize that neuroimagery sharing will be enhanced if data owners are provided with well-defined intermediate levels of data visibility. in this paper, we describe a broadly applicable data sharing model, structured sharing communities (ssc), in which data becomes incrementally visible to communities structured as a complete partial-order; the associated properties of privacy and fairness regulate access to shared data. within ssc, a customized policy space is defined capturing the sharing relationships among specific collaborators.
a general language model for information retrieval. statistical language modeling has been successfully used for speech recognition, part-of-speech tagging, and syntactic parsing. recently, it has also been applied to information retrieval. according to this new paradigm, each document is viewed as a language sample, and a query as a generation process. the retrieved documents are ranked based on the probabilities of producing a query from the corresponding language models of these documents. in this paper, we will present a new language model for information retrieval, which is based on a range of data smoothing techniques, including the good-turning estimate, curve-fitting functions, and model combinations. our model is conceptually simple and intuitive, and can be easily extended to incorporate probabilities of phrases such as word pairs and word triples. the experiments with the wall street journal and trec4 data sets showed that the performance of our model is comparable to that of inquery and better than that of another language model for information retrieval. in particular, word pairs are shown to be useful in improving the retrieval performance.
transformation-based spatial join. spatial join finds pairs of spatial objects having a specific spatial relationship in spatial database systems. a number of spatial join algorithms have recently been proposed in the literature. most of them, however, perform the join in the original space. joining in the original space has a drawback of dealing with sizes of objects and thus has difficulty in developing a formal algorithm that does not rely on heuristics. in this paper, we propose a spatial join algorithm based on the transformation technique. an object having a size in the two-dimensional original space is transformed into a point in the four-dimensional transform space, and the join is performed on these point objects. this can be easily extended to n-dimensional cases. we show the excellence of the proposed approach through analysis and extensive experiments. the results show that the proposed algorithm has a performance generally better than that of the r*-based algorithm proposed by brinkhoff et al. this is a strong indicating that corner transformation preserves clustering among objects and that spatial operations can be performed better in the transform space than in the original space. this reverses the common belief that transformation will adversely affect clustering. we believe that our result will provide a new insight towards transformation-based spatial query processing.
adaptive commitment for distributed real-time transactions. distributed real-time transaction systems are useful for both real-time and high-performance database applications. standard transaction management approaches that use the two-phase commit protocol suffer from its high costs and blocking behavior which is problematic in real-time computing environments. our approach in this paper is to identify ways in which a commit protocol can be made adaptive in the sense that under situations that demand it, such as a transient local overload, the system can dynamically change to a different commitment strategy. the decision to do so can be taken autonomously at any site. the different commitment strategies exploit a trade-off between the cost of commitment and the obtained degree of atomicity. our protocols are based on optimistic commitment strategies, and they rely on local compensatory actions to recover from non-atomic executions. we provide the necessary framework to study the logical and temporal correctness criteria, and we describe examples to illustrate the use of our strategies.
unapparent information revelation: a concept chain graph approach. information generated by multiple authors working independently at different times when analyzed synergistically reveals more information than apparent. for example, a traditional search for connections between the trucking industry and iraqi banks may not produce any documents mentioning both. however, a search that follows trails of associations across documents may suggest a connection between an auto parts manufacturer who exports to iraq, and an iraqi bank providing loans to buy cars. the work described here extends link analysis based on named entities and labeled relationships to general concepts and unnamed associations. unapparent information revelation involves finding chains connecting concepts across documents: it uses a new representation formalism called concept chain graphs.
exploiting syntactic structure of queries in a language modeling approach to ir. natural language processing (nlp) techniques have been explored to enhance the performance of information retrieval (ir) methods with varied results. most efforts in using nlp techniques have been to identify better index terms for representing documents. this use in the indexing phase of ir has implicit effect on retrieval performance. however, the explicit use of nlp techniques during the retrieval or information seeking phase has been restricted to interactive or dialogue systems. recent advances in ir are based on using statistical language models (slm) to represent documents and ranking them based on their model generating a given user query. this paper presents a novel method for using nlp techniques on user queries, specifically, a syntactic parse of a query, in the statistical language modeling approach to ir. in the proposed method, named concept language models, a query is viewed as a sequence of concepts and a concept as a sequence terms. the paper presents different approximations to estimate the concept and term probabilities and compute the query likelihood estimate for documents. some empirical results on trec test collections comparing concept language models with smoothed n-gram language models are presented.
grammar-based task analysis of web logs. the daily use of internet-based services is involved with hundreds of different tasks being performed by multiple users. a single task is typically involved with a sequence of web urls invocation. we study the problem of pattern detection in web logs to identify tasks performed by users, and analyze task trends over time using a grammar-based framework. our results are demonstrated on a corporate intranet portal application with 7000 users over a 6 week period and demonstrate compelling business value from this high-level task analysis.
a compact and efficient image retrieval approach based on border/interior pixel classification. this paper presents \bic (border/interior pixel classification), a compact and efficient cbir approach suitable for broad image domains. it has three main components: (1) a simple and powerful image analysis algorithm that classifies image pixels as either border or interior, (2) a new logarithmic distance (dlog) for comparing histograms, and (3) a compact representation for the visual features extracted from images. experimental results show that the bic approach is consistently more compact, more efficient and more effective than state-of-the-art cbir approaches based on sophisticated image analysis algorithms and complex distance functions. it was also observed that the dlog distance function has two main advantages over vectorial distances (e.g., l1): (1) it is able to increase substantially the effectiveness of (several) histogram-based cbir approaches and, at the same time, (2) it reduces by 50% the space requirement to represent a histogram.
raindrop: a uniform and layered algebraic framework for xqueries on xml streams. xml stream applications bring the challenge of efficiently processing queries on sequentially accessible token-based data. while the automata model is naturally suited for pattern matching on tokenized xml streams, the algebraic model in contrast is a well-established technique for set-oriented processing of self-contained tuples. however, neither automata nor algebraic models are well-equipped to handle both computation paradigms. the goal of the raindrop project is to accommodate these two paradigms within one algebraic framework to take advantage of both. in our query model, both tokenized data and self-contained tuples are supported in a uniform manner. query plans can be flexibly rewritten using equivalence rules to change what computation is done using tokenized data versus tuples. this paper highlights the four abstraction levels in raindrop, namely, semantics-focused plan, stream logical plan, stream physical plan and execution plan. various optimization techniques are provided at each level. the necessity of such a uniform and layered plan is shown by experimental study
information extraction from biomedical literature: methodology, evaluation and an application. journals and conference proceedings represent the dominant mechanisms of reporting new biomedical results. the unstructured nature of such publications makes it difficult to utilize data mining or automated knowledge discovery techniques. annotation (or markup) of these unstructured documents represents the first step in making these documents machine analyzable. in this paper we first present a system called bioannotator for identifying and annotating biological terms in documents. bioannotator uses domain based dictionary look-up for recognizing known terms and a rule engine for discovering new terms. the combination and dictionary look-up and rules result in good performance (87% precision and 94% recall on the genia 1.1 corpus for extracting general biological terms based on an approximate matching criterion). to demonstrate the subsequent mining and knowledge discovery activities that are made feasible by bioannotator, we also present a system called medsummarizer that uses the extracted terms to identify the common concepts in a given group of genes.
web unit mining: finding and classifying subgraphs of web pages. in web classification, most researchers assume that the objects to classify are individual web pages from one or more web sites. in practice, the assumption is too restrictive since a web page itself may not always correspond to a concept instance of some semantic concept (or category) given to the classification task. in this paper, we want to relax this assumption and allow a concept instance to be represented by a subgraph of web pages or a set of web pages. we identify several new issues to be addressed when the assumption is removed, and formulate the web unit mining problem. we also propose an iterative web unit mining (iwum) method that first finds subgraphs of web pages using some knowledge about web site structure. from these web subgraphs, web units are constructed and classified into semantic concepts (or categories) in an iterative manner. our experiments using the webkb dataset showed that iwum improves the overall classification performance and works very well on the more structured parts of a web site.
extracting significant time varying features from text. we propose a simple statistical model for the frequency of occurrence of features in a stream of text. adoption of this model allows us to use classical significance tests to filter the stream for interesting events. we tested the model by building a system and running it on a news corpus. by a subjective evaluation, the system worked remarkably well: almost all of the groups of identified tokens corresponded to news stories and were appropriately placed in time. a preliminary objective evaluation was also used to measure the quality of the system and it showed some of the weaknesses and the power of our approach.