waashk/dblp huggingface dataset:
improving novelty detection for general topics using sentence level information patterns. the detection of new information in a document stream is an important component of many potential applications. in this work, a new novelty detection approach based on the identification of sentence level information patterns is proposed. first, the information-pattern concept for novelty detection is presented with the emphasis on new information patterns for general topics (queries) that cannot be simply turned into specific questions whose answers are specific named entities (nes). then we elaborate a thorough analysis of sentence level information patterns on data from the trec novelty tracks, including sentence lengths, named entities, sentence level opinion patterns. this analysis provides guidelines in applying those patterns in novelty detection particularly for the general topics. finally, a unified pattern-based approach is presented to novelty detection for both general and specific topics. the new method for dealing with general topics will be the focus. experimental results show that the proposed approach significantly improves the performance of novelty detection for general topics as well as the overall performance for all topics from the 2002-2004 trec novelty tracks.
eigen-trend: trend analysis in the blogosphere based on singular value decompositions. the blogosphere - the totality of blog-related web sites - has become a great source of trend analysis in areas such as product survey, customer relationship, and marketing. existing approaches are based on simple counts, such as the number of entries or the number of links. in this paper, we introduce a novel concept, coined eigen-trend, to represent the temporal trend in a group of blogs with common interests and propose two new techniques for extracting eigen-trends in blogs. first, we propose a trend analysis technique based on the singular value decomposition. extracted eigen-trends provide new insights into multiple trends on the same keyword. second, we propose another trend analysis technique based on a higher-order singular value decomposition. this analyzes the blogosphere as a dynamic graph structure and extracts eigen-trends that reflect the structural changes of the blogosphere over time. experimental studies based on synthetic data sets and a real blog data set show that our new techniques can reveal a lot of interesting trend information and insights in the blogosphere that are not obtainable from traditional count-based methods.
database model for web-based cooperative applications. in this paper we propose a model of a database that could become a kernel of cooperative database applications. first, we propose a new data model cdm (collaborative data model) that is oriented for the specificity of multiuser environments, in particular: cooperation scenarios, cooperation techniques and cooperation management. second, we propose to apply to databases supporting collaboration so called multiuser transactions. multiuser transactions are flat transactions in which, in comparison to classical acid transactions, the isolation property is relaxed.
opportunity map: a visualization framework for fast identification of actionable knowledge. data mining techniques frequently find a large number of patterns or rules, which make it very difficult for a human analyst to interpret the results and to find the truly interesting and actionable rules. due to the subjective nature of "interestingness", human involvement in the analysis process is crucial. in this paper, we propose a novel visual data mining framework for the purpose of identifying actionable knowledge quickly and easily from discovered rules and data. this framework is called the opportunity map. it is inspired by some interesting ideas from quality engineering, in particular quality function deployment (qfd) and the house of quality. it associates summarized data or discovered rules with the application objective using an interactive matrix, which enables the user to quickly identify where the opportunities are. the proposed system can be used to visually analyze discovered rules, and other statistical properties of the data. the user can also interactively group actionable attributes and values, and see how they affect the targets of interest. combined with drill-down and comparative analysis, the user can analyze rules and data at different levels of detail. the proposed visualization framework thus represents a systematic and yet flexible method of rule analysis. applications of the system to large-scale data sets from our industrial partner have yielded promising results.
spatial data traversal in road map databases: a graph indexing approach. spatial data are found in geographic information systems such as digital road map databases where city and road attributes are associated with nodes and links in a directed graph. queries on spatial data are expensive because of the recursive property of graph traversal. we propose a graph indexing technique to expedite spatial queries where the graph topology remains relatively stationary. using a probabilistic analysis, this paper shows that the graph indexing technique significantly improves the efficiency of constrained spatial queries.
a case for reconfigurable parallel architectures for information retrieval. as the volume of data and computational requirements of modern information retrieval systems continue to expand, it is inevitable that parallel systems will be necessary to meet these demands. in this research, we provide a conceptual model of a reconfigurable parallel information retrieval system in a multicomputer environment. we develop strategies for scheduling queries in such systems, provide simulation results for implementing these strategies under various different theoretical situations, and present an analytical model of the system behavior.
document quality models for web ad hoc retrieval. the quality of document content, which is an issue that is usually ignored for the traditional ad hoc retrieval task, is a critical issue for web search. web pages have a huge variation in quality relative to, for example, newswire articles. to address this problem, we propose a document quality language model approach that is incorporated into the basic query likelihood retrieval model in the form of a prior probability. our results demonstrate that, on average, the new model is significantly better than the baseline (query likelihood model) in terms of precision at the top ranks.
hybrid index structures for location-based web search. there is more and more commercial and research interest in location-based web search, i.e. finding web content whose topic is related to a particular place or region. in this type of search, location information should be indexed as well as text information. however, the index of conventional text search engine is set-oriented, while location information is two-dimensional and in euclidean space. this brings new research problems on how to efficiently represent the location attributes of web pages and how to combine two types of indexes. in this paper, we propose to use a hybrid index structure, which integrates inverted files and r*-trees, to handle both textual and location aware queries. three different combining schemes are studied: (1) inverted file and r*-tree double index, (2) first inverted file then r*-tree, (3) first r*-tree then inverted file. to validate the performance of proposed index structures, we design and implement a complete location-based web search engine which mainly consists of four parts: (1) an extractor which detects geographical scopes of web pages and represents geographical scopes as multiple mbrs based on geographical coordinates; (2) an indexer which builds hybrid index structures to integrate text and location information; (3) a ranker which ranks results by geographical relevance as well as non-geographical relevance; (4) an interface which is friendly for users to input location-based search queries and to obtain geographical and textual relevant results. experiments on large real-world web dataset show that both the second and the third structures are superior in query time and the second is slightly better than the third. additionally, indexes based on r*-trees are proven to be more efficient than indexes based on grid structures.
efficient retrieval for browsing large image databases. the management of large image databases poses several interesting and challenging problems. these problems range from ingesting the data and extracting meta-data to the efficient storage and retrieval of the data. of particular interest are the retrieval methods and user interactions with an image database during browsing. in image databases, the response to a given query is not an exact well-defined set, rather, the user poses a query and expects a set of responses that should contain many possible candidates from which the user chooses the answer set. we first present the browsing model in alexandria, a digital library for maps and satellite images. designed for content-based retrieval, the relevant information in an image is encoded in the form of a multi-% dimensional feature vector. various techniques have been previously proposed for the efficient retrieval of such vectors by reducing the dimensionality of such vectors. we show that for even moderately large databases (in fact, only 1856 texture images), these approaches do not scale well for exact retrieval. however, as a browsing tool, these dimensionality reduction techniques hold much promise.
ontology-based web site mapping for information exploration. centralized search process requires that the whole collection reside at a single site. this imposes a burden on both the system storage of the site and the network traffic near the site. it thus comes to require the search process to be distributed. recently, more and more web sites provide the ability to search their local collection of web pages. query brokering systems are used to direct queries to the promising sites and merge the results from these sites. creation of meta-information of the sites plays an important role in such systems. in this article, we introduce an ontology-based web site mapping method used to produce conceptual meta-information, the vector space approach, and present a serial of experiments comparing it with na&iuml;ve-bayes approach. we found that the vector space approach produces better accuracy in ontology-based web site mapping.
interval query indexing for efficient stream processing. a large number of continual range queries can be issued against a data stream. usually, a main memory-based query index with a small storage cost and a fast search time is needed, especially if the stream is rapid. in this paper, we present a cei-based query index that meets both criteria for efficient processing of continual interval queries in a streaming environment. this new query index is centered around a set of predefined virtual <i>containment-encoded intervals</i>, or ceis. the ceis are used to first decompose query intervals and then perform efficient search operations. the ceis are defined and labeled such that containment relationships among them are encoded in their ids. the containment encoding makes decomposition and search operations efficient because integer additions and logical shifts can be used to carry out most of the operations. simulations are conducted to evaluate the effectiveness of the cei-based query index and to compare it with alternative approaches. the results show that the cei-based query index significantly outperforms existing approaches in terms of both storage cost and search time.
trajectory queries and octagons in moving object databases. an important class of queries in moving object databases involves trajectories. we propose to divide trajectory predicates into topological and non-topological parts; extend the 9 intersection model of egenhofer-franzosa to a 3-step evaluation strategy for trajectory queries: a filter step, a refinement step, and a tracing step.the filter and refinement steps are similar to region searches. as in spatial databases, approximations of trajectories are typically used in evaluating trajectory queries. in earlier studies, minimum bounding boxes (mbrs) are used to approximate trajectory segments which allow index structures to be built, e.g., tb-trees and r*-trees. the use of mbrs hinders the efficiency since mbrs are very coarse approximations especially for trajectory segments. to overcome this problem, we propose a new type of approximations, "minimum bounding octagon prism" mbop. we extend r*-tree to a new index structure "octagon-prism tree" (op-tree) for mbops of trajectory segments. we conducted experiments to evaluate efficiency of op-trees in performing region searches and trajectory queries. the results show that op-trees improve region searches significantly over synthetic trajectory data sets to tb-trees and r*-trees and can significantly reduce the evaluation cost of trajectory queries compared to tb-trees.
domain-specific keyphrase extraction. document keyphrases provide semantic metadata characterizing documents and producing an overview of the content of a document. they can be used in many text-mining and knowledge management related applications. this paper describes a keyphrase identification program (kip), which extracts document keyphrases by using prior positive samples of human identified domain keyphrases to assign weights to the candidate keyphrases. the logic of our algorithm is: the more keywords a candidate keyphrase contains and the more significant these keywords are, the more likely this candidate phrase is a keyphrase. to obtain prior positive inputs, kip first populates its glossary database using manually identified keyphrases and keywords. it then checks the composition of all noun phrases of a document, looks up the database and calculates scores for all these noun phrases. the ones having higher scores will be extracted as keyphrases.
distributed pagerank computation based on iterative aggregation-disaggregation methods. pagerank has been widely used as a major factor in search engine ranking systems. however, global link graph information is required when computing pagerank, which causes prohibitive communication cost to achieve accurate results in distributed solution. in this paper, we propose a distributed pagerank computation algorithm based on iterative aggregation-disaggregation (iad) method with block jacobi smoothing. the basic idea is divide-and-conquer. we treat each web site as a node to explore the block structure of hyperlinks. local pagerank is computed by each node itself and then updated with a low communication cost with a coordinator. we prove the global convergence of the block jacobi method and then analyze the communication overhead and major advantages of our algorithm. experiments on three real web graphs show that our method converges 5-7 times faster than the traditional power method. we believe our work provides an efficient and practical distributed solution for pagerank on large scale web graphs.
a performance comparison of bitmap indexes. we present a comparison of two new word-aligned schemes with some schemes for compressing bitmap indexes, including the well-known byte-aligned bitmap code (bbc). on both synthetic data and real application data, the new word-aligned schemes use only 50% more space, but perform logical operations on compressed data 12 times faster than bbc. the new schemes achieve this performance advantage by guaranteeing that during logical operations every machine instruction performs useful work on words rather than on bytes or bits as in bbc.
taxonomy-driven computation of product recommendations. recommender systems have been subject to an enormous rise in popularity and research interest over the last ten years. at the same time, very large taxonomies for product classification are becoming increasingly prominent among e-commerce systems for diverse domains, rendering detailed machine-readable content descriptions feasible. amazon.com makes use of an entire plethora of hand-crafted taxonomies classifying books, movies, apparel, and various other goods. we exploit such taxonomic background knowledge for the computation of personalized recommendations. hereby, relationships between super-concepts and sub-concepts constitute an important cornerstone of our novel approach, providing powerful inference opportunities for profile generation based upon the classification of products that customers have chosen. ample empirical analysis, both offline and online, demonstrates our proposal's superiority over common existing approaches when user information is sparse and implicit ratings prevail.
finding more useful information faster from web search results. in this paper, we propose a prototype system for automatic generation of concept hierarchies to be used as an overview of search results. the system sends a user's query to five search engines and receives a returned list of relevant web pages. the system then extracts query-oriented concept terms from snippets that come with the returned hits. concept terms are organized into a concept hierarchy using a co-occurrence-based classification technique. finally, concepts in returned documents are dynamically highlighted according to terms in the selected concept branch that lead to the chosen document. the user study shows that concept hierarchies do provide easy navigation and browsing of web returned documents. the results also show that users can find a document of interest no matter how low it is ranked in the retrieved list.
measuring the meaning in time series clustering of text search queries. we use a combination of proven methods from time series analysis and machine learning to explore the relationship between temporal and semantic similarity in web query logs; we discover that the combination of correlation and cycles is a good, but not perfect, sign of semantic relationship.
efficient query monitoring using adaptive multiple key hashing. monitoring continual queries or subscriptions is to determine the subset of all queries or subscriptions whose predicates match a given event. predicates contain not only equality but also non-equality clauses. event matching is usually accomplished by first identifying a "small" candidate set of subscriptions for an event and then determining the matched subscriptions from the candidate set. prior work has focused on using equality clauses to identify the candidate set. however, we found that completely ignoring non-equality clauses can result in a much larger candidate set. in this paper, we present and evaluate an adaptive multiple key hashing (amkh) method to judiciously include an effective subset of non-equality clauses in candidate set identification. each subscription is mapped to a data point in a multidimensional space based on its predicate clauses. amkh is then used to maintain subscriptions and perform event matching. amkh further provides a controlling mechanism to limit the hash range of a non-equality clause, hence reducing the size of the candidate set. simulations are conducted to study the performance of amkh. the results show that (1) a small number of non-equality clauses can be effectively included by amkh and (2) the attributes whose overall non-equality predicate clauses are most selective should be chosen for inclusion by amkh.
performance thresholding in practical text classification. in practical classification, there is often a mix of learnable and unlearnable classes and only a classifier above a minimum performance threshold can be deployed. this problem is exacerbated if the training set is created by active learning. the bias of actively learned training sets makes it hard to determine whether a class has been learned. we give evidence that there is no general and efficient method for reducing the bias and correctly identifying classes that have been learned. however, we characterize a number of scenarios where active learning can succeed despite these difficulties.
local replication for proxy web caches with hash routing. this paper studies controlled local replication for hash routing, such as carp, among a collection of loosely-coupled proxy web cache servers. hash routing partitions the entire url space among the shared web caches, creating a single logical cache. each partition is assigned to a cache server. duplication of cache contents is eliminated and total incoming traffic to the shared web caches is minimized. client requests for non-assigned-partition objects are forwarded to sibling caches. however, request forwarding increases not only inter-cache traffic but also cpu utilization, thus slows the client response time. we propose a controlled local replication of non-assigned-partition objects in each cache server to effectively reduce the inter-cache traffic. we use a multiple-exit lru to implement controlled local replication. trace-driven simulations are conducted to study the performance impact of local replication. the results show that (1) regardless of cache sizes, with a controlled local replication, the average response time, inter-cache traffic and cpu overhead can be effectively reduced without noticeable increases in incoming traffic; (2) for very large cache sizes, a larger amount of local replication can be allowed to reduce inter-cache traffic without increasing incoming traffic; and (3) local replication is effective even if clients are dynamically assigned to different cache servers.
self-managing technology in ibm db2 universal database. as the cost of both hardware and software falls due to technological advancements and economies of scale, the cost of ownership for database applications is increasingly dominated by the cost of people to manage them. databases are growing rapidly in scale and complexity, while skilled database administrators (dbas) are becoming rarer and more expensive. the scope of responsibility of dbas is indeed daunting. this paper describes the self-managing technology in ibm db2 universal database to illustrate how self-managing technology can enhance the usability of enterprise middleware and reduce the total cost of ownership (tco).
a relational algebra for data/metadata integration in a federated database system. the need for interoperability among databases has increased dramatically with the proliferation of readily available dbms and application software. even within a single organization, data from disparate relational databases must be integrated. a framework for interoperability in a federated system of relational databases should be inherently relational, so that it can use existing techniques for query evaluation and optimization where possible and retain the key features of sql, such as a modest complexity and ease of query formulation. our contribution is a logspace relational algebra, the meta-algebra (ma), for data/metadata integration among relational databases containing semantically similar information in schematically disparate formats. the ma is a simple yet powerful extension of the classical relational algebra (ra). the ma has a natural declarative counterpart, the meta-query language (mql), which we briefly describe. we state a result showing mql and the ma are computationally equivalent, which enables us to algebratize mql queries in fundamentally the same way as ordinary sql queries. this algebratization in turn enables us to use ma equivalences to facilitate the application of known query optimization techniques to mql query evaluation.