waashk/dblp huggingface dataset:
web-dl: an experience in building digital libraries from the web. the web contains a huge volume of information, almost all unstructured and, therefore, difficult to manage. in digital libraries, however, information is explicitly organized, described, and managed. in this paper, we propose an architecture that allows the construction of digital libraries from the web, using standard protocols and archival technologies, and incorporating powerful digital library and data extraction tools, thus benefiting from the breadth of the web contents, but supporting services and organization available in digital libraries. the proposed architecture was applied to the networked digital library of theses and dissertations, providing an important first step toward rapid construction of large dls from the web, as well as a large-scale solution for interoperability between independent digital libraries.
f4: large-scale automated forecasting using fractals. forecasting has attracted a lot of research interest, with very successful methods for periodic time series. here, we propose a fast, automated method to do non-linear forecasting, for both periodic as well as chaotic time series. we use the technique of delay coordinate embedding, which needs several parameters; our contribution is the automated way of setting these parameters, using the concept of `intrinsic dimensionality'. our operational system has fast and scalable algorithms for preprocessing and, using r-trees, also has fast methods for forecasting. the result of this work is a black-box which, given a time series as input, finds the best parameter settings, and generates a prediction system. tests on real and synthetic data show that our system achieves low error, while it can handle arbitrarily large datasets.
searching web databases by structuring keyword-based queries. on-line information services have become widespread in the web nowadays. however, web users are non-specialized and have a great variety of interests. thus, interfaces for web databases must be simple and uniform. in this paper we present an approach, based on bayesian networks, for querying web databases using keywords only. according to this approach, the user inputs a query through a simple search-box interface. from the input query, one or more plausible structured queries are derived and submitted to web databases. the results are then retrieved and presented to the user as ranked answers. our approach reduces the complexity of existing on-line interfaces and offers a solution to the problem of querying several distinct web databases with a single interface. the applicability of the proposed approach was demonstrated by experimental results with 3 databases, obtained with a prototype search system that implements it. we have found that from 77% to 95% of the time, one of the top three resulting structured queries is the proper one. further, when the user selects one of these three top queries for processing, the ranked answers present average precision figures from 60% to about 100%.
rapid association rule mining. association rule mining is a well-researched area where many algorithms have been proposed to improve the speed of mining. in this paper, we propose an innovative algorithm called rapid association rule mining (rarm) to once again break this speed barrier. it uses a versatile tree structure known as the support-ordered trie itemset (sotrieit) structure to hold pre-processed transactional data. this allows rarm to generate large 1-itemsets and 2-itemsets quickly without scanning the database and without candidate 2-itemset generation. it achieves significant speed-ups because the main bottleneck in association rule mining using the apriori property is the generation of candidate 2-itemsets. rarm has been compared with the classical mining algorithm apriori and it is found that it outperforms apriori by up to two orders of magnitude (100 times), much more than what recent mining algorithms are able to achieve.
performance and implications of semantic indexing in a distributed environment. a research prototype is presented for semantic indexing and retrieval in information retrieval. the prototype is motivated by a desire to provide a more efficient and effective information retrieval system compared to the current state of the art. an overview of the interspace architecture layers is discussed. an object model supporting semantic operations is developed. the model contains a rich set of classes and relationships of the data for the semantic indexing module. the basis of our semantic indexing is done by the creation of concept space. a concept space is an index of a collection that uses document statistics to capture the relationships between concepts. it is useful for boosting text search, by term suggestion of alternative terms semantically related to query terms. over the years, we have developed generic technology for concept spaces computation on large collections across many subjects. recent computations on discipline-scale collections have been made on high-end supercomputers. this paper describes our implementation and implications of the computation in a distributed computing environment. experimental results using different collection sizes and number of processes are presented to show the feasibility of this approach. we also show that laboratory and community collections are already easily computable using a group of pcs in a lab via a message-passing model. we conclude that pc clusters will shortly be able to compute semantic indexes for any real collections.
analytical version control management in a hypertext system. in this paper it is shown how structural and cognitive versioning issues can be efficiently managed in a petri nets based hypertextual model. the advantages of this formalism are enhanced by modular and structured modeling; modularity allows to focus the attention only on some modules, while giving the abstraction of the others. each module owns metaknowledge that is useful in defining new layers and contexts.the central point of the data model is the formulation and resolution of three recurrence equations, effective in describing both the versioning and the derivation history; these equations permit to express in precise terms both the structural evolution (changes operated on specific nodes of the net) and the behavioral one (changes concerning browsing).
facilitating transformations in a human genome project database. human genome project databases present a confluence of interesting database challenges: rapid schema and data evolution, complex data entry and constraint management, and the need to integrate multiple data sources and software systems which range over a wide variety of models and formats. while these challenges are not necessarily unique to biological databases, their combination, intensity and complexity are unusual and make automated solutions imperative. we illustrate these problems in the context of the philadelphia genome center for human chromosome 22, and describe a new approach to a solution for these problems, by means of a deductive language for expressing database transformations and constraints.
digital annotation of printed documents. we present a general model and information server for the digital annotation of printed documents. the resulting annotation framework supports both informal and structured annotations as well as context-dependent services. a demonstrator application for mammography that features both enhanced writing and reading activities is described.
a music recommendation system based on music data grouping and user interests. with the growth of the world wide web, a large amount of music data is available on the internet. in addition to searching expected music objects for users, it becomes necessary to develop a recommendation service. in this paper, we design the music recommendation system (mrs) to provide a personalized service of music recommendation. the music objects of midi format are first analyzed. for each polyphonic music object, the representative track is first determined, and then six features are extracted from this track. according to the features, the music objects are properly grouped. for users, the access histories are analyzed to derive user interests. the content-based, collaborative and statistics-based recommendation methods are proposed, which are based on the favorite degrees of the users to the music groups. a series of experiments are carried out to show that our approach is feasible.
replication and retrieval strategies of multidimensional data on parallel disks. aside from enhancing data availability during disk failures, replication of data is also used to speed up i/o performance of read-intensive applications. there are two issues that need to be addressed: (a) data placement (which disks should store the copies of each data block?) and (b) scheduling (given a query q, and a placement scheme p of the data, from which disk should each block in q be retrieved so that retrieval time is minimized?) in this paper, we consider range queries and assume that the dataset is a multidimensional grid and r copies of each unit block of the grid must be stored among m disks. to accurately measure performance of a scheduling algorithm, we consider a metric that takes into account the scheduling overhead as well as the time it takes to retrieve the data blocks from the disks. we describe several combinations of data placement schemes and scheduling algorithms and analyze their performance for range queries with respect to the above metric. we then present simulation results for the most interesting case r=2, showing that the strategies do perform better than the previously known method, especially for large queries.
on the efficient evaluation of relaxed queries in biological databases. in this paper, a new technique is developed to support the query relaxation in biological databases. query relaxation is required due to the fact that queries tend not to be expressed exactly by the users, especially in scientific databases such as biological databases, in which complex domain knowledge is heavily involved. to treat this problem, we propose the concept of the so-called fuzzy equivalence classes to capture important kinds of domain knowledge that is used to relax queries. this concept is further integrated with the canonical techniques for pattern searching such as the position tree and automaton theory. as a result, fuzzy queries produced through relaxation can be efficiently evaluated. this method has been successfully utilized in a practical biological database - the gpcrdb.
mass: a multi-axis storage structure for large xml documents. effective indexing for xml must consider both the query requirements of the xpath language and the dynamic nature of xml. we introduce mass, a multiple axis storage structure, to provide scalable indexing for xpath expressions with guaranteed update performance. we describe the building blocks of mass and provide results that demonstrate mass's scalability. we show that mass can outperform other state-of-the-art xml indexing solutions, even with constrained system resources.
xkvalidator: a constraint validator for xml. the role of xml in data exchange is evolving from one of merely conveying the structure of data to one that also conveys its semantics. in particular, several proposals for key and foreign key constraints have recently appeared, and aspects of these proposals have been adopted within xmlschema.in this paper, we examine the problem of checking keys and foreign keys in xml documents using a validator based on sax. the algorithm relies on an indexing technique based on the paths found in key definitions, and can be used for checking the correctness of an entire document (bulk checking) as well as for checking updates as they are made to the document (incremental checking). the asymptotic performance of the algorithm is linear in the size of the document or update. furthermore, experimental results demonstrate reasonable performance.
using conjunction of attribute values for classification. advances in the efficient discovery of frequent itemsets have led to the development of a number of schemes that use frequent itemsets to aid developing accurate and efficient classifiers. these approaches use the frequent itemsets to generate a set of composite features that expand the dimensionality of the underlying dataset. in this paper, we build upon this work and (i) present a variety of schemes for composite feature selection that achieve a substantial reduction in the number of features without adversely affecting the accuracy gains, and (ii) show (both analytically and experimentally) that the composite features can lead to improved classification models even in the context of support vector machines, in which the dimensionality can automatically be expanded by the use of appropriate kernel functions.
statistical relationship determination in automatic thesaurus construction. statistical relationship determination among terms is one of the key issues in automatic thesaurus construction. we systematically analyze existing relevant approaches based on their underlying probabilistic assumptions, and propose a combined approach that overcomes their limitations.
i/o-efficient techniques for computing pagerank. over the last few years, most major search engines have integrated link-based ranking techniques in order to provide more accurate search results. one widely known approach is the pagerank technique, which forms the basis of the google ranking scheme, and which assigns a global importance measure to each page based on the importance of other pages pointing to it. the main advantage of the pagerank measure is that it is independent of the query posed by a user; this means that it can be precomputed and then used to optimize the layout of the inverted index structure accordingly. however, computing the pagerank measure requires implementing an iterative process on a massive graph corresponding to billions of web pages and hyperlinks.in this paper, we study i/o-efficient techniques to perform this iterative computation. we derive two algorithms for pagerank based on techniques proposed for out-of-core graph algorithms, and compare them to two existing algorithms proposed by haveliwala. we also consider the implementation of a recently proposed topic-sensitive version of pagerank. our experimental results show that for very large data sets, significant improvements over previous results can be achieved on machines with moderate amounts of memory. on the other hand, at most minor improvements are possible on data sets that are only moderately larger than memory, which is the case in many practical scenarios.
local methods for estimating pagerank values. the google search engine uses a method called pagerank, together with term-based and other ranking techniques, to order search results returned to the user. pagerank uses link analysis to assign a global importance score to each web page. the pagerank scores of all the pages are usually determined off-line in a large-scale computation on the entire hyperlink graph of the web, and several recent studies have focused on improving the efficiency of this computation, which may require multiple hours on a workstation. however, in some scenarios, such as online analysis of link evolution and mining of large web archives such as the internet archive, it may be desirable to quickly approximate or update the pageranks of individual nodes without performing a large-scale computation on the entire graph. we address this problem by studying several methods for efficiently estimating the pagerank score of a particular web page using only a small subgraph of the entire web. in our model, we assume that the graph is accessible remotely via a link database (such as the altavista connectivity server) or is stored in a relational database that performs lookups on disks to retrieve node and connectivity information. we show that a reasonable estimate of the pagerank value of a node is possible in most cases by retrieving only a moderate number of nodes in the local neighborhood of the node.
regularizing ad hoc retrieval scores. the cluster hypothesis states: closely related documents tend to be relevant to the same request. we exploit this hypothesis directly by adjusting ad hoc retrieval scores from an initial retrieval so that topically related documents receive similar scores. we refer to this process as score regularization. score regularization can be presented as an optimization problem, allowing the use of results from semi-supervised learning. we demonstrate that regularized scores consistently and significantly rank documents better than unregularized scores, given a variety of initial retrieval algorithms. we evaluate our method on two large corpora across a substantial number of topics.
an optimal graph traversal algorithm for evaluating linear binary-chain programs. grahne et al. have presented a graph algorithm for a subset of recursive queries. this method consists of two phases. in the first phase, the method transforms a linear binary-chain program into a set of equations over expressions containing predicate symbols. in the second phase, a graph is constructed from the equations and the answers are produced by traversing the relevant paths. here we describe a new algorithm which requires less time than the algorithm of grahne et al. the key idea of the improvement is to reduce the search space that will be traversed when a query is invoked. further, we speed up the evaluation of cyclic data by generating most answers directly in terms of the answers already found and the associated &ldquo;path information&rdquo; instead of traversing the corresponding paths as usual. in this way, our algorithm achieves a linear time complexity for both cyclic and non-cyclic data.
clustermap: labeling clusters in large datasets via visualization. with the rapid increase of data in many areas, clustering on large datasets has become an important problem in data analysis. since cluster analysis is a highly iterative process, cluster analysis on large datasets prefers short iteration on a relatively small representative set. thus, a two-phase framework "sampling/summarization - iterative cluster analysis" is often applied in practice. since the clustering result only labels the small representative set, there are problems with extending the result to the entire large dataset, which are almost ignored by the traditional clustering research. this extending is often named as labeling process. labeling irregular shaped clusters, distinguishing outliers and extending cluster boundary are the main problems in this stage. we address these problems and propose a visualization-based approach to dealing with them precisely. this approach partially involves human into the process of defining and refining the structure "clustermap". based on this structure, the clustermap algorithm scans the large dataset to adapt the boundary extension and generate the cluster labels for the entire dataset. experimental result shows that clustermap can preserve cluster quality considerably with low computational cost, compared to the distance-comparison-based labeling algorithms.
swoogle: a search and metadata engine for the semantic web. swoogle is a crawler-based indexing and retrieval system for the semantic web. it extracts metadata for each discovered document, and computes relations between documents. discovered documents are also indexed by an information retrieval system which can use either character n-gram or urirefs as keywords to find relevant documents and to compute the similarity among a set of documents. one of the interesting properties we compute is <i>ontology rank</i>, a measure of the importance of a semantic web document.
categorizing information objects from user access patterns. many web sites have dynamic information objects whose topics change over time. classifying these objects automatically and promptly is a challenging and important problem for site masters. traditional content-based and link structure based classification techniques have intrinsic limitations for this task. this paper proposes a framework to classify an object into an existing category structure by analyzing the users' traversals in the category structure. the key idea is to infer an object's topic from the predicted preferences of users when they access the object. we compare two approaches using this idea. one analyzes collective user behavior and the other each user's accesses. we present experimental results on actual data that demonstrate a much higher prediction accuracy and applicability with the latter approach. we also analyze the correlation between classification quality and various factors such as the number of users accessing the object. to our knowledge, this work is the first effort in combining object classification with user access prediction.
time weight collaborative filtering. collaborative filtering is regarded as one of the most promising recommendation algorithms. the item-based approaches for collaborative filtering identify the similarity between two items by comparing users' ratings on them. in these approaches, ratings produced at different times are weighted equally. that is to say, changes in user purchase interest are not taken into consideration. for example, an item that was rated recently by a user should have a bigger impact on the prediction of future user behaviour than an item that was rated a long time ago. in this paper, we present a novel algorithm to compute the time weights for different items in a manner that will assign a decreasing weight to old data. more specifically, the users' purchase habits vary. even the same user has quite different attitudes towards different items. our proposed algorithm uses clustering to discriminate between different kinds of items. to each item cluster, we trace each user's purchase interest change and introduce a personalized decay factor according to the user own purchase behaviour. empirical studies have shown that our new algorithm substantially improves the precision of item-based collaborative filtering without introducing higher order computational complexity.
evaluating window joins over punctuated streams. we explore join optimizations in the presence of both time-based constraints (sliding windows) and value-based constraints (punctuations). we present the first join solution named pwjoin that exploits such combined constraints to shrink the runtime join state and to propagate punctuations to benefit downstream operators. we design a state structure for pwjoin that facilitates the exploitation of both constraint types. we also explore optimizations enabled by the interactions between window and punctuation, e.g., early punctuation propagation. the costs of the pwjoin are analyzed using a cost model. we also conduct an experimental study using cape continuous query system. the experimental results show that in most cases, by exploiting punctuations, pwjoin outperforms the pure window join with regard to both memory overhead and throughput. our technique complements the joins in the literature, such as symmetric hash join or window join, to now require less runtime resources without compromising the accuracy of the result.
expedite: a system for encoded xml processing. as xml becomes an increasingly popular format for information exchange, the efficient processing of broadcast xml data on a constrained device (for example, a cell phone or a pda) becomes a critical task. in this paper we present the expedite system: a new model of data processing in an information exchange environment, which "migrates" the power of the data-sending server to receivers for efficient processing. it consists of a simple and general encoding scheme for servers, and streaming query processing algorithms on encoded xml stream for data receivers with constrained computing abilities. experiments show the impressive performance of expedite.