waashk/dblp huggingface dataset:
a practical system of keyphrase extraction for web pages. keyphrases can be used to facilitate web users grasping the main topic(s) of a web page. we present a practical system of automatic keyphrase extraction for web pages. in this system, a regression model was first trained based on a set of human-labeled documents. then it was used to extract keyphrases from new pages automatically. this paper makes three contributions. first, the structure information in a web page was investigated for keyphrase extraction task. second, the query log data associated with a web page collected by a search engine server were used to help keyphrase extraction. third, a method was put forward in this paper in order to evaluate the similarity of phrases.
an optimized implementation for vml based on pattern matching and dynamic programming. in an object-oriented database system (oodbs), objects exist persistently and object i/o is transparent to the programmer. therefore, some mechanism in the system must initiate i/o as the program runs. in this paper we present an approach based on pattern matching and dynamic programming that allows a program to interact efficiently with the runtime storage layer. we are interested in allowing programs to manipulate very large objects without necessarily reading them entirely. if a program touches only a small part of a large object, the problem is how to determine the part of the object needed. in this paper, we present an approach based on pattern matching and dynamic programming to resolve this problem.we discuss and solve this problem in the context of vml, a modeling language of an open object-oriented database language. the vml compiler translates vml programs into c++ programs which contain calls to the object manager. we provide a detailed description of our implementation with the hope that our approach will foster the development of object-oriented database systems based on c++.
practical evaluation of ir within automated classification systems. this paper describes some of the work we have done to evaluate and compare the use of three ir systems (verity, lsi, and smart) as black boxes within an automated classification environment. we use automated classification to make a quantitative comparison of the effectiveness of the systems within this context. in so doing, we also develop criteria for the construction of a useful training set. these results lead to metrics useful in the integration of ir systems into larger applications. we conclude with an initial api for an ir component within an automated classification architecture.
an extended logic programming based multi-agent system formalization in mobile environments. in this paper, we propose an extended logic programming based formalization for a multi-agent system in mobile environments. such a system consists of a number of agents connected via wire or wireless communication channels. our formalization is knowledge oriented and has declarative semantics inherited from extended logic programming. this model can be used to study the details of knowledge transaction in mobile environments.
auto-generation of topic hierarchies for web images from users' perspectives. in this paper, we propose an approach to automatically generating a yahoo!-like topic hierarchy for organizing web images from users' perspectives. relatively little effort has been devoted towards providing such a taxonomy simultaneously considering users' image requests for semantic and visual information. based on the characteristic that a web-image query may be refined by various attributes, the proposed approach hierarchically groups similar queries from search engine logs into topic classes at different semantic levels. the generated topic hierarchy has the advantages of organizing image data from users' perspectives for browsing, searching, annotation and users' needs analysis.a series of experiments have been conducted on real-world image search engine logs. experimental results show that the proposed approach is feasible to generate topic hierarchies for web images. moreover, the generated hierarchy has been successfully applied to analysis of users' search interests, which have more focuses on some specific domains when compared with document requests.
a pattern matching language for spatio-temporal databases. we propose a pattern matching language for spatio-temporal databases. the matching process in time dimension is based upon the evolutionary nature of time, but in spatial dimension it is based on placement, shape and sizes of regions. the concept of pattern matching introduced in this paper is independent of the choice of the underlying model for spatio-temporal databases. in particular, the pattern matching language seamlessly extends our sql-like query language parasql for spatio-temporal databases. the pattern matching language would also have application in active databases, because patterns can be used as triggers.
database support for species extraction from the biosystematics literature: a feasibility demonstration. a part of the biosystematics literature is currently being digitized and manually marked up with xml. fast search on such documents shall be feasible. but marking up such documents incurs high costs, and biologists would like to know the value of such an activity in advance. deploying standard xml database technology in a straightforward way is not feasible, because of two characteristics of biosystematics documents. the first one is that descriptions of taxa are related, i.e., a more specific taxon should inherit from a more general one. the combination of inheritance with information-retrieval mechanisms gives rise to difficulties addressed in this article. the second issue is the frequent occurrence of very specific technical terms in such documents, i.e., geographical information or biological terms. to investigate the characteristics of the search in the presence of such difficulties, we have designed and implemented a respective system, based on relational database technology. we use a collection of xml documents that mimics the characteristics of biosystematics documents, as we will explain. we propose two query-evaluation alternatives and compare them by means of performance experiments. it turns out that our techniques can administer the envisioned corpus of documents efficiently and cope with those problems at the same time.
requirement-based data cube schema design. on-line analytical processing (olap) requires efficient processing of complex decision support queries over very large databases. it is well accepted that pre-computed data cubes can help reduce the response time of such queries dramatically. a very important design issue of an efficient olap system is therefore the choice of the right data cubes to materialize. we call this problem the data cube schema design problem. in this paper we show that the problem of finding an optimal data cube schema for an olap system with limited memory is np-hard. as a more computationally efficient alternative, we propose a greedy approximation algorithm cmp and its variants. algorithm cmp consists of two phases. in the first phase, an initial schema consisting of all the cubes required to efficiently answer the user queries is formed. in the second phase, cubes in the initial schema are selectively merged to satisfy the memory constraint. we show that cmp is very effective in pruning the search space for an optimal schema. this leads to a highly efficient algorithm. we report the efficiency and the effectiveness of cmp via an empirical study using the tpc-d benchmark. our results show that the data cube schemas generated by cmp enable very efficient olap query processing.
word segmentation and recognition for web document framework. it is observed that a better approach to web information understanding is to base on its document framework, which is mainly consisted of (i) the title and the url name of the page, (ii) the titles and the url names of the web pages that it points to, (iii) the alternative information source for the embedded web objects, and (iv) its linkage to other web pages of the same document. investigation reveals that a high percentage of words inside the document framework are &ldquo;compound words&rdquo; which cannot be understood by ordinary dictionaries. they might be abbreviations or acronyms, or concatenations of several (partial) words. to recover the content hierarchy of web documents, we propose a new word segmentation and recognition mechanism to understand the information derived from the web document framework. a maximal bi-directional matching algorithm with heuristic rules is used to resolve ambiguous segmentation and meaning in compound words. an adaptive training process is further employed to build a dictionary of recognisable abbreviations and acronyms. empirical results show that over 75% of the compound words found in the web document framework can be understood by our mechanism. with the training process, the success rate of recognising compound words can be increased to about 90%.
mailrank: using ranking for spam detection. can we use social networks to combat spam? this paper investigates the feasibility of mailrank, a new email ranking and classification scheme exploiting the social communication network created via email interactions. the underlying email network data is collected from the email contacts of all mailrank users and updated automatically based on their email activities to achieve an easy maintenance. mailrank is used to rate the sender address of arriving emails such that emails from trustworthy senders can be ranked and classified as spam or non-spam. the paper presents two variants: basic mailrank computes a global reputation score for each email address, whereas in personalized mailrank the score of each email address is different for each mailrank user. the evaluation shows that mailrank is highly resistant against spammer attacks, which obviously have to be considered right from the beginning in such an application scenario. mailrank also performs well even for rather sparse networks, i.e., where only a small set of peers actually take part in the ranking of email addresses.
processing xpath queries with xml summaries. range labeling and structural joins are well-studied techniques for efficiently processing xpath queries. however, when xpath queries become long, many times of structural joins are required. to solve this problem, we developed a method to reduce the number of joins and nodes read from the disk using strong dataguides. our method can process single paths without any joins and twig patterns with joins amongst branching nodes and leaves in queries. experimental results verified that our approach outperforms the best optimization technique for structural joins by factors of up to several hundreds of times.
node ranking in labeled directed graphs. our work is motivated by the problem of ranking hyper-linked documents for a given query. given an arbitrary directed graph with edge and node labels, we present a new flow-based model and an efficient method to dynamically rank the nodes of this graph with respect to any of the original labels. ranking documents for a given query in a hyper-linked document set and ranking of authors/articles for a given topic in a citation database are some typical applications of our method. we outline the structural conditions that the graph must satisfy for our ranking to be different from the traditional <i>pagerank</i>. we have built a system using two indices that is capable of dynamically ranking documents for any given query. we validate our system and method using experiments on a few datasets: a crawl of the ibm intranet (12 million pages), a crawl of the <b>www</b> (30 million pages) and the dblp citation dataset. we compare our method to existing schemes for topic-biased ranking that require a classifier and the traditional <i>pagerank</i>. in these experiments, we demonstrate that our method is well suited for fine-grained ranking and that our method performs better than the existing schemes. we also demonstrate that our system can obtain an improved ranking with very little impact on query time.
the liberal media and right-wing conspiracies: using cocitation information to estimate political orientation in web documents. this paper introduces a simple method for estimating <i>cultural orientation</i>, the affiliation of online entities in a polarized field of discourse. in particular, cocitation information is used to estimate the political orientation of hypertext documents. a type of cultural orientation, the political orientation of a document is the degree to which it participates in traditionally left- or right-wing beliefs. estimating documents' political orientation is of interest for personalized information retrieval and recommender systems. in its application to politics, the method uses a simple probabilistic model to estimate the strength of association between a document and left- and right-wing communities. the model estimates the likelihood of cocitation between a document of interest and a small number of documents of known orientation. the model is tested on three sets of data, 695 partisan web documents, 162 political weblogs, and 72 non-partisan documents. accuracy above 90% is obtained from the cocitation model, outperforming lexically based classifiers at statistically significant levels.
dynamic and hierarchical spatial access method using integer searching. dynamic and complex computation in the area of geographic information system (gis) or mobile computing system involves huge amount of spatial objects such as points, boxes, polygons, etc and requires a scalable data structure and an efficient management tool for this information. in this paper, for a dynamic management of spatial objects, we construct a hierarchical dynamic data structure, called an ist/opg hierarchy, which may overcome some limitations of existing spatial access methods (sams). the hierarchy is constructed by combining three primary components: (1) minimum boundary rectangle (mbr), which is the most widely used method among sams; (2) the population-based domain slicing, which is modified from the grid file [14]; (3) extended optimal integer searching algorithm [4]. for dynamic management of spatial objects in the ist/opg hierarchy, a number of primary and supplementary operations are introduced. this paper includes a comparative analysis of our approach with previous sams, such as r-tree, r+-tree and r*-tree and qsf-tree. the results of analysis show that our approach is better than other sams in construction and query time and space requirements. specifically, for a given search domain with n objects, our query operations yield $o($ \scriptsize $\sqrt {\frac {\log n} {\log\log n}}$\normalsize $)$ compared to $o(\log n)$ of the fast sam and an ist/opg hierarchy containing $n$ objects can be constructed in $o(n$ \scriptsize $\sqrt {\frac {\log n}{\log\log n}}$\normalsize $)$ time and o(n) space.
computing consistent query answers using conflict hypergraphs. a consistent query answer in a possibly inconsistent database is an answer which is true in every (minimal) repair of the database. we present here a practical framework for computing consistent query answers for large, possibly inconsistent relational databases. we consider relational algebra queries without projection, and denial constraints. because our framework handles union queries, we can effectively (and efficiently) extract indefinite disjunctive information from an inconsistent database. we describe a number of novel optimization techniques applicable in this context and summarize experimental results that validate our approach.
alternative representations and abstractions for moving sensors databases. moving sensors refers to an emerging class of data intensive applications that inpacts disciplines such as communication, health-care, scientific applications, etc. these applications consist of a fixed number of sensors that move and produce streams of data as a function of time. they may require the system to match these streams against stored streams to retrieve relevant data (patterns). with communication, for example, a speaking impaired individual might utilize a haptic glove that translates hand signs into written (spoken) words. the glove consists of sensors for different finger joints. these sensors report their location and values as a function of time, producing streams of data. these streams are matched against a repository of spatio-temporal streams to retrieve the corresponding english character or word.the contributions of this study are two fold. first, it introduces a framework to store and retrieve "moving sensors" data. the framework advocates physical data independence and software-reuse. second, we investigate alternative representations for storage and retrieve of data in support of query processing. we quantify the tradeoff associated with these alternatives using empirical data robocup soccer matches.
the verity federated infrastructure. in the course of researching a subject, it is often necessary to submit the same search request to multiple heterogeneous information sources in order to (a) aggregate as much information as possible, and (b) integrate different aspects of the subject into a coherent report. while it is clear that there is value in providing a federated search solution to make dealing with multiple sources less time-consuming, not all organizations aggregate from the same sources, and once the information has been retrieved, not all organizations want them to be integrated in the same way.the verity federated infrastructure addresses this problem by providing a flexible framework for adding new sources and customizing the way in which results are integrated, post-processed and presented. a new source is made available by writing a java module called a worker that abides by the search interface of the source. sources can range from simple information feeds to more complex applications, e.g., crm systems, relational databases, etc. workers also perform post-processing on the results returned by other workers, e.g., to provide uniform scores for results from different sources, filtering, etc. this post-processing enables different results to be integrated into a coherent report. post-processing is triggered by events that propagate between workers and is done asynchronously in the background while results are being viewed. this ability to do background post-processing allows execution of time-consuming operations that provide substantial value without adversely affecting user experience. finally, search results are returned and viewed incrementally, which enables searching of peer-to-peer networks via peer workers that we have developed.
operational requirements for scalable search systems. prior research into search system scalability has primarily addressed query processing efficiency [1, 2, 3] or indexing efficiency [3], or has presented some arbitrary system architecture [4]. little work has introduced any formal theoretical framework for evaluating architectures with regard to specific operational requirements, or for comparing architectures beyond simple timings [5] or basic simulations [6, 7]. in this paper, we present a framework based upon queuing network theory for analyzing search systems in terms of operational requirements. we use response time, throughput, and utilization as the key operational characteristics for evaluating performance. within this framework, we present a scalability strategy that combines index partitioning and index replication to satisfy a given set of requirement.
automatically classifying database workloads. the type of the workload on a database management system (dbms) is a key consideration in tuning the system. allocations for resources such as main memory can be very different depending on whether the workload type is online transaction processing (oltp) or decision support system (dss). in this paper, we present an approach to automatically identifying a dbms workload as either oltp or dss. we build a classification model based on the most significant workload characteristics that differentiate oltp from dss, and then use the model to identify any change in the workload type. we construct a workload classifier from the browsing and ordering profiles of the tpc-w benchmark. experiments with an industry-supplied workload show that our classifier accurately identifies the mix of oltp and dss work within an application workload.
a practical web-based approach to generating topic hierarchy for text segments. it is crucial in many information systems to organize short text segments, such as keywords in documents and queries from users, into a well-formed topic hierarchy. in this paper, we address the problem of generating topic hierarchies for diverse text segments with a general and practical approach that uses the web as an additional knowledge source. unlike long documents, short text segments typically do not contain enough information to extract reliable features. this work investigates the possibilities of using highly ranked search-result snippets to enrich the representation of text segments. a hierarchical clustering algorithm is then applied to create the hierarchical topic structure of text segments. different from traditional clustering algorithms, which tend to produce cluster hierarchies with a very unnatural shape, the approach tries to produce a more natural and comprehensive hierarchy. extensive experiments were conducted on different domains of text segments. the obtained results have shown the potential of the proposed approach, which is believed able to benefit many information systems.
frequent pattern discovery with memory constraint. we explore in this paper a practicably interesting mining task to retrieve frequent itemsets with memory constraint. as opposed to most previous works that concentrate on improving the mining efficiency or on reducing the memory size by best effort, we first attempt to constrain the upper memory size that can be utilized by mining frequent itemsets in this paper.
a multiple-resolution method for edge-centric data clustering. recent works in spatial data clustering view the input data set in terms of inter-point edge lengths rather than the points themselves. cluster detection in such a system is a matter of finding connected paths of edges whose weight is no greater than some user input threshold or cutoff value. the smtin algorithm[9] is one such system that uses delaunay triangulation to compute the set of nearest neighbor edges quickly and efficiently. experiments demonstrate a substantial performance and accuracy improvement using smtin in comparison to other clustering systems.the resolution of the clusters discovered in the smtin system is directly related to the choice of a cutoff threshold, which makes smtin perform poorly for input sets with clusters at multiple resolutions. in this work we introduce an edge-centric clustering method that detects clusters at multiple resolutions. our algorithm detects differences in density among groups of points and uses multiple cutoff points in order to account for clusters at different resolutions. one of the main benefits of the multi-resolution approach of our system is the ability to accurately cluster points that other systems would consider to be noise. experiments indicate a substantial improvement in the clustering quality of our system in comparison to smtin as well as the removal of the requirement of an input distance-threshold, achieved with comparable theoretical as well as actual runtime performance. we present promising directions for this new algorithm.