waashk/dblp huggingface dataset:
improving intranet search-engines using context information from databases. information in enterprises comes in documents and data bases. from a semantic viewpoint, both kinds of information are usually tightly connected. in this paper, we propose to enhance common search-engines with contextual information retrieved from databases. we establish system requirements and anecdotally demonstrate how documents and database information can be represented as the nodes of a graph. then, we give an example how we exploit this graph information for document retrieval.
clustering high-dimensional data using an efficient and effective data space reduction. this paper introduces a new algorithm for clustering data in high-dimensional feature spaces, called gardenhd. the algorithm is organized around the notion of data space reduction, i.e. the process of detecting dense areas (dense cells) in the space. it performs effective and efficient elimination of empty areas that characterize typical high-dimensional spaces and an efficient adjacency-connected agglomeration of dense cells into larger clusters. it produces a compact representation that can effectively capture the essence of data. gardenhd is a hybrid of cell-based and density-based clustering. however, unlike typical clustering methods in its class, it applies a recursive partition of sparse regions in the space using a new space-partitioning strategy. the properties of this partitioning strategy greatly facilitate data space reduction. the experiments on synthetic and real data sets reveal that gardenhd and its data space reduction are effective, efficient, and scalable.
recent developments in text summarization. with the explosion in the quantity of on-line text and multimedia information in recent years, demand for text summarization technology is growing. increased pressure for technology advances is coming from users of the web, on-line information sources, and new mobile devices, as well as from the need for corporate knowledge management. commercial companies are increasingly starting to offer text summarization capabilities, often bundled with information retrieval tools. in this paper, i will discuss the significance of some recent developments in summarization technology.
joint optimization of cost and coverage of query plans in data integration. existing approaches for optimizing queries in data integration use decoupled strategies--attempting to optimize coverage and cost in two separate phases. since sources tend to have a variety of access limitations, such phased optimization of cost and coverage can unfortunately lead to expensive planning as well as highly inefficient plans. in this paper we present techniques for joint optimization of cost and coverage of the query plans. our algorithms search in the space of parallel query plans that support multiple sources for each subgoal conjunct. the refinement of the partial plans takes into account the potential parallelism between source calls, and the binding compatibilities between the sources included in the plan. we start by introducing and motivating our query plan representation. we then briefly review how to compute the cost and coverage of a parallel plan. next, we provide both a system-r style query optimization algorithm as well as a greedy local search algorithm for searching in the space of such query plans. finally we present a simulation study that demonstrates that the plans generated by our approach will be significantly better, both in terms of planning cost, and in terms of plan execution cost, compared to the existing approaches.
ordinal association rules for error identification in data sets. a new extension of the boolean association rules, ordinal association rules, that incorporates ordinal relationships among data items, is introduced. one use for ordinal rules is to identify possible errors in data. a method that finds these rules and identifies potential errors in data is proposed.
mining coverage statistics for websource selection in a mediator. recent work in data integration has shown the importance of statistical information about the coverage and overlap of sources for efficient query processing. despite this recognition there are no effective approaches for learning the needed statistics. the key challenge in learning such statistics is keeping the number of needed statistics low enough to have the storage and learning costs manageable. naive approaches can become infeasible very quickly. in this paper we present a set of connected techniques that estimate the coverage and overlap statistics while keeping the needed statistics tightly under control. our approach uses a hierarchical classification of the queries, and threshold based variants of familiar data mining techniques to dynamically decide the level of resolution at which to learn the statistics. we describe the details of our method, and present experimental results demonstrating the efficiency of the learning algorithms and the effectiveness of the learned statistics.
a statistical model for scientific readability. in this paper, we present a new method of using statistical models to estimate readability [1]. language model is used to capture the content information. it is combined with linguistic feature model by a linear form. experiments show that this new method has a better performance than the widely used flesch-kincaid readability formula.
knowledge discovery in patent databases. in our days the business, scientific and personal databases are growing in an exponential rate. however, what is truly valuable is the knowledge that can be extracted from the stored data. knowledge discovery in patent databases was traditionally based on manual analysis carried out from statistical experts. nowadays the increasing interest of many actors have led to the development of new tools for discovering and exploiting information related to technological activities and innovation, "hidden" in patent databases. in this paper we present a system that combines efficient and innovative methodologies and tools for the analysis of patent data stored in international databases and the production of scientific and technological indicators.
on the storage and retrieval of continuous media data. continuous media applications, which require a guaranteed transfer rate of the data, are becoming an integral part of daily computational life. however, conventional file systems do not provide rate guarantees, and are therefore not suitable for the storage and retrieval of continuous media data (e.g., audio, video). to meet the demands of these new applications, continuous media file systems, which provide rate guarantees by managing critical storage resources such as memory and disks, must be designed.in this paper, we highlight the issues in the storage and retrieval of continuous media data. we first present a simple scheme for concurrently retrieving multiple continuous media streams from disks. we then introduce a a clever allocation technique for storing continuous media data that eliminates disk latency and thus, drastically reduces ram requirements. we present, for video data, schemes for implementing the operations fast-forward, rewind and pause. finally, we conclude by outlining directions for future research in the storage and retrieval of continuous media data.
a model for weighting image objects in home photographs. the paper presents a contribution to image indexing consisting in a weighting model for visible objects -- or image objects -- in home photographs. to improve its effectiveness this weighting model has been designed according to human perception criteria about what is estimated as important in photographs. four basic hypotheses related to human perception are presented, and their validity is estimated as compared to actual observations from a user study. finally a formal definition of this weighting model is presented and its consistence with the user study is evaluated.
on structuring formal, semi-formal and informal data to support traceability in systems engineering environments. the development of large, complex systems poses a number of challenges for systems engineers, not least of which is the ability to ensure user requirements have been satisfied. effective requirements management - an amalgam of information capture, information <i>storage</i> and <i>management</i>, and information <i>dissemination</i> activities - is crucial in that respect. in this paper we concentrate on one of the core issues of <i>information management</i> in a requirements management context - namely <i>traceability</i>. traceability is the common term for mechanisms to record and navigate relationships between artifacts produced by development processes. however, realising effective traceability in systems engineering environments is complicated by the fact that engineers use a range of notations to describe complex systems. these range from natural language (informal), to graphical notations such as statecharts (semi-formal) to languages with a well defined (formal) semantics such as vdm-sl and spark ada. most have tool support, although a lack of well-defined approaches to integration leads to inconsistencies and limits traceability between their respective data sets (internal models). this paper demonstrates an approach based on meta-modelling that enables traceability links to be established and consistency maintained between tools.
infer: a relational query language without the complexity of sql. the infer query language allows users to express queries without referencing relations or specifying joins. since the infer syntax is similar to but less restrictive than sql, users can easily write highly expressive queries that are automatically completed by infer's inference engine. infer's sql-based syntax is familiar to current database users, and its improved ranking and query explanation system makes it easier to use.
task-oriented world wide web retrieval by document type classification. this paper proposes a novel approach to accurately searching web pages for relevant information in problem solving by specifying a web document category instead of the user's task. accessing information from world wide web pages as an approach to problem solving has become commonplace. however, such a search is difficult with current search services, since these services only provide keyword-based search methods that are equivalent to narrowing down the target references according to domains. however, problem solving usually involves both a domain and a task. accordingly, our approach is based on problem solving tasks. to specify a user's problem solving task, we introduce the concept of document types that directly relate to the problem solving tasks; with this approach, users can easily designate problem solving tasks. we implemented pagetypesearch system based on our approach. classifier of pagetypesearch classifies web pages into the document types by comparing their pages with typical structural characteristics of the types. we compare pagetypesearch using the document typeindices with a conventional keyword-based search system in experiments. the average precision of the document type-based search is 88.9%, while the average precision of the keyword-based search is 31.2%. moreover, the number of irrelevant references gathered by our system is about one-thirteenth that of traditional keyword-based search systems. our approach has practical advantages for problem solving by introducing the viewpoint of tasks to achieve higher performance.
"geoplot": spatial data mining on video libraries. are "tornado" touchdowns related to "earthquakes"? how about to "floods", or to "hurricanes"? in informedia [14], using a gazetteer on news video clips, we map news onto points on the globe and find correlations between sets of points. in this paper we show how to find answers to such questions, and how to look for patterns on the geo-spatial relationships of news events. the proposed tool is "geoplot", which is fast to compute and gives a lot of useful information which traditional text retrieval can not find.we describe our experiments on 2-year worth of video data (~ 20 gbytes). there we found that geoplot can find unexpected correlations that text retrieval would never find, such as those between "earthquake" and "volcano", and "tourism" and "wine".in addition, geoplot provides a good visualization of a data set's characteristics. characteristics at all scales are shown in one plot and a wealth of information is given, for example, geo-spatial clusters, characteristic scales, and intrinsic (fractal) dimensions of the events' locations.
incremental evaluation of a monotone xpath fragment. this paper shows a scheme for incremental evaluation of xpath queries. here, we focus on a monotone fragment of xpath, i.e., when a data is deleted from (or inserted to) the database, only deletion (insertion, resp.) may occur to query answers. for efficiently processing deletions, we store information on partial matchings, i.e., which elements were participating in matchings for which query answers, and also store counters showing how many matchings each query answer had. we use the information on the partial matchings also for skipping a part of computation upon data insertion. we investigate properties of the xpath fragment in order to keep the amount of information we store as small as possible.
the enosys markets data integration platform: lessons from the trenches. enosys markets offers a state-of-the-art data integration software platform to support the development of the next generation of ebusiness applications that deliver value by providing new levels of function for customer relationship management, e-commerce, supply chain management, and decision support. these applications require that data be integrated from information sources that exist both within and across organizational boundaries. the enosys markets data integration architecture and product family provides a complete end-to-end xml-based solution for integrating and querying distributed information sources. it incorporates advanced research into xml and database technology. we present the product architecture and components, discuss the key technical challenges, and outline the technical concepts and innovations employed in the enosys platform.
lattice-based tagging using support vector machines. tagging algorithms have become increasingly important for identifying lexical and semantic features of unstructured text. we describe an approach to lattice-based tagging that estimates joint transition and emission probabilities using support vector machines. the technique offers several advantages over alternative methods, including the ability to accommodate non-local features, support for hundreds of thousands of features, and language-neutrality. we demonstrate the technique on two tagging applications: named entity recognition and part-of-speech tagging.
caching constrained mobile data. as mobile devices get ubiquitous and grow in computational power, their management of interdependent data also becomes increasingly important. the mobile environment exhibits all the characteristics of a distributed database plus the feature of whimsical connectivity. consequently, transactions respecting data consistency can suffer unbounded and unpredictable delays at both mobile and stationary nodes. the currently popular multi-tier model, in which mobile devices are in one end and always-connected stationary servers in the other, has certain practical advantages. however, it assumes that all integrity constraints are evaluated at the servers and hence relies on the semantics of operations for any autonomy enhancement of the mobile devices. in this paper, we examine the idea of constraint localization in cases where two mobile nodes each own data that share a constraint. it relies on reformulation of a constraint into more flexible local constraints that give more autonomy to the mobile nodes. the scheme also involves dynamic changes of these local constraints through negotiation, which we call re-localization. to overcome the problem of simultaneous requests for such re-localization, we give algorithms along with experimental results indicating their effectiveness.
a unified environment for fusion of information retrieval approaches. prior work has shown that combining results of various retrieval approaches and query representations can improve search effectiveness. today, many meta-search engines exist which combine the results of various search engines in the hopes of improving overall effectiveness. however, the combination of results from different search engines masks variations in parsers, and other indexing techniques (stemming, stop words, etc.) this makes it difficult to assess the utility of the fusion technique. we have implemented the two most prevalent retrieval strategies: probabilistic and vector space using the same parser and the same relational retrieval engine. first, we identified a model that enables the fusion of an arbitrary number of sources. next, we tested various linear combinations of these two methods as well as various thresholds for identifying retrieved documents. our results show some improvement of effectiveness, but they also provide us for a baseline from which we can continue with other retrieval strategies and test the effect of fusing these strategies.
unified utility maximization framework for resource selection. this paper presents a unified utility framework for resource selection of distributed text information retrieval. this new framework shows an efficient and effective way to infer the probabilities of relevance of all the documents across the text databases. with the estimated relevance information, resource selection can be made by explicitly optimizing the goals of different applications. specifically, when used for database recommendation, the selection is optimized for the goal of high-recall (include as many relevant documents as possible in the selected databases); when used for distributed document retrieval, the selection targets the high-precision goal (high precision in the final merged list of documents). this new model provides a more solid framework for distributed information retrieval. empirical studies show that it is at least as effective as other state-of-the-art algorithms.
a dimensionality reduction technique for efficient similarity analysis of time series databases. efficiently searching for similarities among time series and discovering interesting patterns is an important and non-trivial problem with applications in many domains. the high dimensionality of the data makes the analysis very challenging. to solve this problem, many dimensionality reduction methods have been proposed. pca (piecewise constant approximation) and its variant have been shown efficient in time series indexing and similarity retrieval. however, in certain applications, too many false alarms introduced by the approximation may reduce the overall performance dramatically. in this paper, we introduce a new piecewise dimensionality reduction technique that is based on vector quantization. the new technique, pvqa (piecewise vector quantized approximation), partitions each sequence into equi-length segments and uses vector quantization to represent each segment by the closest (based on a distance metric) codeword from a codebook of key-sequences. the efficiency of calculations is improved due to the significantly lower dimensionality of the new representation. we demonstrate the utility and efficiency of the proposed technique on real and simulated datasets. by exploiting prior knowledge about the data, the proposed technique generally outperforms pca and its variants in similarity searches.
dist: a distributed spatio-temporal index structure for sensor networks. we consider the general problem of tracking moving objects in sensor networks. the specific application we consider is that of tracking a chemical plume moving over a large infrastructure network. we present a distributed index structure dist that stores and updates distributed summaries as the plume moves. we present algorithms for range queries on the history of the plume. dist localizes information with respect to time and space using a hierarchy that scales with the plume size. the highlight of our work is an analytical model to predict the cost of query algorithms based on the query location, query size, and plume's spatio-temporal distribution. using this model, our adaptive scheme chooses the optimal scheme. experimental results show that dist outperforms alternative techniques in query, update, and storage costs, and scales well with the number of plumes.
using bi-modal alignment and clustering techniques for documents and speech thematic segmentations. in this paper, we describe a new method for a simultaneous thematic segmentation of the meeting dialogs and the documents discussed or visible throughout the meeting. this bi-modal method is suitable for multimodal applications that are centered on documents, such as meetings and lectures, where documents can be aligned with meeting dialogs. bringing into play this alignment, our bi-modal segmentation method first transforms its results into a set of nodes in a 2d graph space, where the two axes represent respectively the document units and the meeting dialogs units. secondly, via a clustering method, the most connected regions in the constituted bi-graph are detected. finally, the denser clusters are projected on the two axes. the two sequences of segments, obtained on both axes, represent the thematic structure of the document and of the meeting dialogs respectively. we present in this article this bi-modal segmentation technique and its performance compared with two mono-modal segmentation methods.
prefix-querying: an approach for effective subsequence matching under time warping in sequence databases. this paper discusses an index-based subsequence matching that supports time warping in large sequence databases. time warping enables finding sequences with similar patterns even when they are of different lengths. in our earlier work, we suggested an efficient method for whole matching under time warping. this method constructs a multi-dimensional index on a set of feature vectors, which are invariant to time warping, from data sequences. for filtering at feature space, it also applies a lower-bound function, which consistently underestimates the time warping distance as well as satisfies the triangular inequality.in this paper, we incorporate the prefix-querying approach based on sliding windows into the earlier approach. for indexing, we extract a feature vector from every subsequence inside a sliding window and construct a multi-dimensional index using a feature vector as indexing attributes. for query processing, we perform a series of index searches using the feature vectors of qualifying query prefixes. our approach provides effective and scalable subsequence matching even with a large volume of a database. we also prove that our approach does not incur false dismissal. to verify the superiority of our method, we perform extensive experiments. the results reveal that our method achieves significant speedup with real-world s&p 500 stock data and with very large synthetic data.
context modeling and discovery using vector space bases. in this paper, context is modeled by vector space bases and its evolution is modeled by linear transformations from one base to another. each document or query can be associated to a distinct base, which corresponds to one context. also, algorithms are proposed to discover contexts from document, query or groups or them. linear algebra can thus by employed in a mathematical framework to process context, its evolution and application.