waashk/dblp huggingface dataset:
graph-based object-oriented approach for structural and behavioral representation of multimedia data. the management of multimedia information poses special requirements for multimedia information systems. both representation and retrieval of the complex and multifaceted multimedia data are not easily handled with the flat relational model and require new data models. in the last several years, object-oriented and graph-based data models are actively pursued approaches for handling the multimedia information. in this paper the characteristics of the novel graph-based object-oriented data model are presented. this model represents the structural and behavioral aspects of data that form multimedia information systems. it also provides for handling the continuously changing user requirements and the complexity of the schema and data representation in multimedia information systems using the schema versioning approach and perspective version abstraction.
mining the web for answers to natural language questions. the web is now becoming one of the largest information and knowledge repositories. many large scale search engines (google, fast, northern light, etc.) have emerged to help users find information. in this paper, we study how we can effectively use these existing search engines to mine the web and discover the "correct" answers to factual natural language questions.we propose a probabilistic algorithm called qasm (question answering using statistical models) that learns the best query paraphrase of a natural language question. we validate our approach for both local and web search engines using questions from the trec evaluation. we also show how this algorithm can be combined with another algorithm (ansel) to produce precise answers to natural language questions.
summarization evaluation using relative utility. we present a series of experiments to demonstrate the validity of relative utility (ru) as a measure for evaluating extractive summarizers. ru is applicable in both single-document and multi-document summarization, is extendable to arbitrary compression rates with no extra annotation effort, and takes into account both random system performance and interjudge agreement. our results using the jhu summary corpus indicate that ru is a reasonable and often superior alternative to several common evaluation metrics.
knowledge discovery from texts: a concept frame graph approach. we address the text content mining problem through a concept based framework by constructing a conceptual knowledge base and discovering knowledge therefrom. defining a novel representation called the concept frame graph (cfg), we propose a learning algorithm for constructing a cfg knowledge base from text documents. an interactive concept map visualization technique is presented for user-guided knowledge discovery from the knowledge base. through experimental studies on real life documents, we observe that the proposed approach is promising for mining deeper knowledge.
the edam project: mining mass spectra and more. the edam project is a collaborative effort between computer scientists and environmental chemists at carleton college and uw-madison. the goal is to develop data mining techniques for advancing the state of the art in analyzing atmospheric aerosol datasets. the traditional approach for particle measurement, which is the collection of bulk samples of particulates on filters, is not adequate for studying particle dynamics and real-time correlations. this has led to the development of a new generation of real-time instruments that provide continuous or semi-continuous streams of data about certain aerosol properties. however, these instruments have added a significant level of complexity to atmospheric aerosol data, and dramatically increased the amounts of data to be collected, managed, and analyzed. we are investigating techniques for automatically labeling mass spectra from different kinds of aerosol mass spectrometers, and then analyzing and exploring the rich spatiotemporal information collected from multiple geographically distributed instruments. in this talk, i will present an overview of some novel data mining problems, describe some of the techniques we are developing to address them, and discuss the broader applicability of these techniques to problems from other domains.
a structure-sensitive framework for text categorization. this paper presents a framework called structure sensitive categorization(sscat), that exploits document structure for improved categorization. there are two parts to this framework, viz. (1) documents often have layout structure, such that logically coherent text is grouped together into fields using some mark-up language. we use a log-linear model, which associates one or more features with each field. weights associated with the field features are learnt from training data and these weights quantify the per-class importance of the field features in determining the category for the document. (2) we employ a technique that exploits the parse tree of fields that are phrasal constructs, such as title and associates weights with words in these constructs while boosting weights of important words called focus words. these weights are learnt from example instances of phrasal constructs, marked with the corresponding focus words. the learning is accomplished by training a classifier that uses linguistic features obtained from the text's parse structure. the weighted words, in fields with phrasal constructs, are used in obtaining features for the corresponding fields in the overall framework. sscat was tested on the supervised categorization task of over one million products from yahoo!'s on-line shopping data. with an accuracy of over 90%, our classifier outperforms naive bayes and support vector machines. this not only shows the effectiveness of sscat but also strengthens our belief that linguistic features based on natural language structure can improve tasks such as text categorization.
techniques for efficient fragment detection in web pages. the existing approaches to fragment-based publishing, delivery and caching of web pages assume that the web pages are manually fragmented at their respective web sites. however manual fragmentation of web pages is expensive, error prone, and not scalable. this paper proposes a novel scheme to automatically detect and flag possible fragments in a web site. our approach is based on an analysis of the web pages dynamically generated at given web sites with respect to their information sharing behavior, personalization characteristics and change patterns.
structural features in content oriented xml retrieval. the structural features of xml components are an extra source of information that should be used in a content-oriented retrieval task on this type of documents. in this paper we explore one of the structural features from the inex collection [1] that could be used in content-oriented search. we analyse the gain this knowledge could add to the performance of an information retrieval system and present a first approach on how this structural information could be extracted from a relevance feedback process to be used as priors in a language modelling framework.
approaches to collection selection and results merging for distributed information retrieval. we have investigated two major issues in distributed information retrieval (dir), namely: collection selection and search results merging. while most published works on these two issues are based on pre-stored metadata, the approaches described in this paper involve extracting the required information at the time the query is processed. in order to predict the relevance of collections to a given query, we analyse a limited number of full documents (e.g., the top five documents) retrieved from each collection and then consider term proximity within them. on the other hand, our merging technique is rather simple since input only requires document scores and lengths of results lists. our experiments evaluate the retrieval effectiveness of these approaches and compare them with centralised indexing and various other dir techniques (e.g., cori). we conducted our experiments using two testbeds: one containing news articles extracted from four different sources (2 gb) and another containing 10 gb of web pages. our evaluations demonstrate that the retrieval effectiveness of our simple approaches is worth considering.
towards data warehouse design. this paper focuses on data warehouse modelling. the conceptual model we defined, is based on object concepts extended with specific concepts like generic classes, temporal classes and archive classes. the temporal classes are used to store the detailed evolutions and the archive classes store the summarised data evolutions. we also provide a flexible concept allowing the administrator to define historised parts and non-historised parts into the warehouse schema. moreover, we introduce constraints which configure the data warehouse behaviour and these various parts. to validate our propositions, we describe a prototype dedicated to the data warehouse design.
exploiting hierarchical relationships in conceptual search. as the number of available web pages grows, users experience increasing difficulty finding documents relevant to their interests. one of the underlying reasons for this is that most search engines find matches based on keywords, regardless of their meanings. to provide the user with more useful information, we need a system that disambiguates queries by including information about the user's conceptual framework. this is the goal of keyconcept, a conceptual search engine. during indexing, keyconcept automatically classifies documents into concepts selected from a reference concept hierarchy. during retrieval, keyconcept ranks documents based on a combination of keyword and conceptual similarity. this paper describes the system architecture and discusses the results of experiments that evaluate the effect of exploiting the hierarchical relationships between concepts during retrieval. our results confirm that conceptual match significantly improves the precision of the search results over keyword match alone. in addition, the use of the concept hierarchy to prune irrelevant search results also significantly increases precision.
efficient runtime generation of association rules. mining frequent patterns in transaction databases has been a popular subject in data mining research. common activities include finding patterns in database transactions, times-series, and exceptions. the apriori algorithm is a widely accepted method of generating frequent patterns. the algorithm can require many scans of the database and can seriously tax resources. new methods of finding association rules, such as the frequent pattern tree (fp-tree) have improved performance, but still have problems when new data becomes available and require two scans of the database.this paper proposes a new method, which requires only one scan of the database and supports update of patterns when new data becomes available. we design a new structure called pattern repository (pr), which stores all of the relevant information in a highly compact form and allows direct derivation of the fp-tree and association rules quickly with a minimum of resources. in addition, it supports run-time generation of association rules by considering only those patterns that meet on-line data requirements.
a vertical distance-based outlier detection method with local pruning. "one person's noise is another person's signal". outlier detection is used to clean up datasets and also to discover useful anomalies, such as criminal activities in electronic commerce, computer intrusion attacks, terrorist threats, agricultural pest infestations, etc. thus, outlier detection is critically important in the information-based society. this paper focuses on finding outliers in large datasets using distance-based methods. first, to speedup outlier detections, we revise knorr and ng's distance-based outlier definition; second, a vertical data structure, instead of traditional horizontal structures, is adopted to facilitate efficient outlier detection further. we tested our methods against national hockey league dataset and show an order of magnitude of speed improvement compared to the contemporary distance-based outlier detection approaches.
the robustness of content-based search in hierarchical peer to peer networks. hierarchical <i>peer to peer</i> networks with multiple directory services are an important architecture for large-scale file sharing due to their effectiveness and efficiency. recent research argues that they are also an effective method of providing large-scale content-based federated search of text-based digital libraries. in both cases the directory services are critical resources that are subject to attack or failure, but the latter architecture may be particularly vulnerable because content is less likely to be replicated throughout the network. this paper studies the robustness, effectiveness and efficiency of content-based federated search in hierarchical <i>peer to peer</i> networks when directory services fail unexpectedly. several recovery methods are studied using simulations with varying failure rates. experimental results show that quality of service and efficiency degrade gracefully as the number of directory service failures increases. furthermore, they show that content-based search mechanisms are more resilient to failures than the match-based search techniques.
a lock method for kbmss using abstraction relationships' semantics. knowledge base management systems (kbmss) are a growing research area finding applicability in different domains. as a consequence, the demand for ever-larger knowledge bases (kbs) is growing more and more. inside this context, knowledge sharing turns out to be a crucial point to be supported by kbmss. in this paper, we propose a way of controlling knowledge sharing. we show how we obtain serializability of transactions providing many different locking granules, which are based on the semantics of the abstraction relationships. the main benefit of our technique is the high degree of potential concurrency, to be obtained through a logical partitioning of the kb graph and the provision of lock types used for each referenced partition. by this way, we capture more of the semantics contained in a kb graph, through an interpretation of its edges grounded in the abstraction relationships, and make feasible a full exploitation of all inherent parallelism in a knowledge representation approach.
extracting semi-structured data through examples. in this paper, we describe an innovative approach to extracting semi-structured data from web sources. the idea is to collect a couple of example objects from the user and to use this information to extract new objects from new pages or texts. to perform the extraction of new objects, we introduce a bottom-up extration strategy and, through experimentation, demonstrate that it works quite effectively with distinct web sources, even if only a few examples are provided by the user.
of parts and relationships: an unending quest. systems biology is a field which focuses on the interpretation of large, diverse sets of biological measurements in order to elucidate the complex mechanisms that underly important and (seemingly simple) macroscopic phenotypes. the problem at hand is hierarchical in nature, with the hierarchy spanning many levels. each of these levels can be thought of as comprising multiple active agents that are diverse in their nature (e.g. genes, proteins, pathways, organelles, etc) and also in their behavior. it is within this setting that one seeks to build an integrated view of the system under study, as soon as the relevant units and the complex inter- and intra-level relationships in which these units participate have been characterized. implicit in the above outline are the assumptions that a) a complete and presumably correct list of parts exists for the system that is being studied; and, b) most, if not all, of the relevant relationships involving these parts are available. through the research work of my group and of others, there is increasing evidence that the situation is likely to be more complicated than initially estimated, and that one should be watchful when it comes to making or relying on the above two assumptions. in fact, more surprising and currently undiscovered things may be lurking in the ?genomics? box: support for this possibility will be provided through the brief summaries of recent advances that we have made in diverse areas such as association discovery, gene discovery, horizontal gene transfer and rna interference. throughout this quest, repositories of biological information will continue to remain our guiding light, whereas time-honored computational methods will continue to be the mainstay of our arsenal.
simple bm25 extension to multiple weighted fields. this paper describes a simple way of adapting the bm25 ranking formula to deal with structured documents. in the past it has been common to compute scores for the individual fields (e.g. title and body) independently and then combine these scores (typically linearly) to arrive at a final score for the document. we highlight how this approach can lead to poor performance by breaking the carefully constructed non-linear saturation of term frequency in the bm25 function. we propose a much more intuitive alternative which weights term frequencies <i>before</i> the non-linear term frequency saturation function is applied. in this scheme, a structured document with a title weight of two is mapped to an unstructured document with the title content repeated twice. this more verbose unstructured document is then ranked in the usual way. we demonstrate the advantages of this method with experiments on reuters vol1 and the trec dotgov collection.
high-performing feature selection for text classification. this paper reports a controlled study on a large number of filter feature selection methods for text classification. over 100 variants of five major feature selection criteria were examined using four well-known classification algorithms: a naive bayesian (nb) approach, a rocchio-style classifier, a k-nearest neighbor (knn) method and a support vector machine (svm) system. two benchmark collections were chosen as the testbeds: reuters-21578 and small portion of reuters corpus version 1 (rcv1), making the new results comparable to published results. we found that feature selection methods based on chi2 statistics consistently outperformed those based on other criteria (including information gain) for all four classifiers and both data collections, and that a further increase in performance was obtained by combining uncorrelated and high-performing feature selection methods.the results we obtained using only 3% of the available features are among the best reported, including results obtained with the full feature set.
quality of service transferred to information retrieval: the adaptive information retrieval system. users often quit an information retrieval system (ir system) very frustrated, because they cannot find the information matching their information needs. we have identified the following two main reasons: too high expectations and wrong use of the system. our approach which addresses both issues is based on the transfer of the concept of quality of service to ir systems: the user first negotiates the retrieval success rates with the ir system, so he knows what to expect from the system in advance. second, by dynamic adaptation to the retrieval context, the ir system tries to improve the user's queries and thereby tries to exploit the underlying information source as best as possible.
hierarchical classification as an aid to database and hit-list browsing. a navigational aid for databases that relies on unsupervised hierarchical classification is presented. the approach to hierarchical classification, based on both functional and topological features, supports the creation of deep hierarchies in which succeeding levels represent increasing degrees of abstraction. this allows the user to quickly evaluate the result of a query and to locate interesting items and classes of items by performing a tree traversal rather than a sequential perusal of a hit list or a series of ad hoc query refinements. in very large databases where classical querying methods are increasingly inadequate such as chemical reaction databases, such a browsing method is required in order to manage the flood of information with which the user is confronted.
structural extraction from visual layout of documents. most information extraction systems focus on the textual content of the documents. they treat documents as sequences or of words, disregarding the physical and typographical layout of the information.. while this strategy helps in focusing the extraction process on the key semantic content of the document, much valuable information can also be derived form the document physical appearance. often, fonts, physical positioning and other graphical characteristics are used to provide additional context to the information. this information is lost with pure-text analysis. in this paper we describe a general procedure for structural extraction, which allows for automatic extraction of entities from the document based on their visual characteristics and relative position in the document layout. our structural extraction procedure is a learning algorithm, which knows how to automatically generalizes from examples. the procedure is a general one, applicable to any document format with visual and typographical information. we also then describe a specific implementation of the procedure to pdf documents, called pes (pdf extraction system). pes works with pdf documents and is able to extract such fields such as author(s), title, date, etc. with very high accuracy.
teg: a hybrid approach to information extraction. this paper describes a hybrid statistical and knowledge-based information extraction model, able to extract entities and relations at the sentence level. the model attempts to retain and improve the high accuracy levels of knowledge-based systems while drastically reducing the amount of manual labor by relying on statistics drawn from a training corpus. the implementation of the model, called teg (trainable extraction grammar), can be adapted to any ie domain by writing a suitable set of rules in a scfg (stochastic context free grammar) based extraction language, and training them using an annotated corpus. the system does not contain any purely linguistic components, such as pos tagger or parser. we demonstrate the performance of the system on several named entity extraction and relation extraction tasks. the experiments show that our hybrid approach outperforms both purely statistical and purely knowledge-based systems, while requiring orders of magnitude less manual rule writing and smaller amount of training data. the improvement in accuracy is slight for named entity extraction task and more pronounced for relation extraction.
document release versus data access controls: two sides of the same coin?. the database and document worlds have traditionally had different approaches to security. databases provide access controls on structured data, while document security interrogates the outgoing information, based on document markings and actual contents. for the emerging world in which many documents are generated from structured data (and vice versa), the separation can cause failure, implementation-dependence, inconsistency, and wasted effort. after comparing approaches and mechanisms in the two areas, we identify issues in security administration and implementation in military and medical applications. we then present elements of a unifying model.