waashk/dblp huggingface dataset:
interactive mining and knowledge reuse for the closed-itemset incremental-mining problem. using concept lattices as a theoretical background for finding association rules [11] has led to designing algorithms like charm [10], close [7] or closet [8]. while they are considered as extremely appropriate when finding concepts for association rules, due to the smaller amount of results, they do not cover a certain area of significant results, namely the pseudo-intents that form the base for global implications. we have proposed an approach that, besides finding all proper partial implications, also finds the pseudo-intents. the way our algorithm is devised, it allows certain important operations on concept lattices, like adding or extracting items, meaning we can reuse previously found results. it is a well-known fact that mining association rules may lead to a large amount of results. since, the mining results are meant to be understood by the user, we have come to the conclusion that he will benefit more from starting small, with some of the items in the data base, understand a small amount of results, and then add items receiving only the extra-results. this way the number of human interventions during the "full" mining process is increased and the process becomes user-driven.
multi-relational data mining: an introduction. data mining algorithms look for patterns in data. while most existing data mining approaches look for patterns in a single data table, multi-relational data mining (mrdm) approaches look for patterns that involve multiple tables (relations) from a relational database. in recent years, the most common types of patterns and approaches considered in data mining have been extended to the multi-relational case and mrdm now encompasses multi-relational (mr) association rule discovery, mr decision trees and mr distance-based methods, among others. mrdm approaches have been successfully applied to a number of problems in a variety of areas, most notably in the area of bioinformatics. this article provides a brief introduction to mrdm, while the remainder of this special issue treats in detail advanced research topics at the frontiers of mrdm.
comparison of graph-based and logic-based multi-relational data mining. we perform an experimental comparison of the graph-based multi-relational data mining system, subdue, and the inductive logic programming system, cprogol, on the mutagenesis dataset and various artificially generated bongard problems. experimental results indicate that subdue can significantly outperform cprogol while discovering structurally large multi-relational concepts. it is also observed that cprogol is better at learning semantically complicated concepts and it tends to use background knowledge more effectively than subdue. an analysis of the results indicates that the differences in the performance of the systems are a result of the difference in the expressiveness of the logic-based and the graph-based representations. the ability of graph-based systems to learn structurally large concepts comes from the use of a weaker representation whose expressiveness is intermediate between propositional and first-order logic. the use of this weaker representation is advantageous while learning structurally large concepts but it limits the learning of semantically complicated concepts and the utilization background knowledge.
multi-relational data mining 2004: workshop report. in this report we briefly review the 3rd workshop on multi-relational data mining (mrdm-2004), which was organized by the authors and held in seattle, wa, on august 22, as part of the workshop program of the tenth acm sigkdd international conference on knowledge discovery and data mining. the goal of the workshop was to bring together researchers and practitioners of data mining and interested in methods and applications of finding patterns in expressive languages from multi-relational, complex, and/or structured data.
the 5th international workshop on multimedia data mining (mdm/kdd2004). in this short report we summarize the presentations, conclusions and directions of future work that were discussed during mdm/kdd2004 workshop, held in conjunction with the 11th acm sigkdd international conference on knowledge discovery and data mining, august 22, 2004 at seattle, washington, usa.
multi-relational data mining: a workshop report. in this report, we briefly review the multi-relational data mining workshop, which was held in edmonton, canada on july, 23, 2002 as part of the workshop program of the 8th acm sigkdd international conference on knowledge discovery and data mining (kdd-02).
kdd-2006 workshop report: theory and practice of temporal data mining. in this paper we provide a summary of the workshop on theory and practice of temporal data mining held in conjunction with acm sigkdd 2006, on august 20st in philadelphia, pennsylvania, usa. we report in detail about the research issues addressed in the talks at the workshop.
multirelational data mining 2003: workshop report. in this report, we briefly review the second international workshop on multi-relational data mining (mrdm-03), which was organized by the authors and held in washington, d.c. on august 27th, 2003 as part of the workshop program of the ninth acm sigkdd international conference on knowledge discovery and data mining (kdd-03). the goal of the workshop was to bring together researchers and practitioners of data mining and interested in methods and applications of finding patterns in expressive languages from multi-relational, complex and/or structured data.
one class svm for yeast regulation prediction. in this paper, we outline the main steps leading to the development of the winning solution for task 2 of kdd cup 2002 (yeast gene regulation prediction). our unusual solution was a pair of linear classifiers in high dimensional space (&sim;14,000), developed with just 38 and 84 training examples, respectively, all belonging to the target class only. the classifiers were built using the support vector machine approach outlined in the paper.
combining data and text mining techniques for yeast gene regulation prediction: a case study. in order to solve task 2 of the kdd cup 2002, we exploited various available information sources. in particular, use of relational information describing the interactions among genes and information automatically extracted from scientific abstracts improves the accuracy of our predictions.
service quality improvement through business process management based on data mining. to improve the quality of service, internal business processes should be managed like the intermediate products are controlled for quality of final products in the manufacturing industry. the business process management (bpm) aims to improve processes and requires both analysis and evaluation of practices. in this paper, we introduce the voice of call center customers (voc) as a data source for bpm in the service industry. we adopt a voc management framework that acquires data about business processes performance and quality of services. we develop a web-based system for analyzing the voc of a life insurance company, which helps decision makers understand customer needs better and helps them make consistent decisions regarding customer support. it uses conventional statistical and data mining techniques to identify customer voice patterns. we gather actual customer complaints from the service operation of the target company. using this data, the system pinpoints problematic areas where complaints happened, the relationship among problems, and the root cause of problems.
why so many clustering algorithms: a position paper. we argue that there are many clustering algorithms, because the notion of "cluster" cannot be precisely defined. clustering is in the eye of the beholder, and as such, researchers have proposed many induction principles and models whose corresponding optimization problem can only be approximately solved by an even larger number of algorithms. therefore, comparing clustering algorithms, must take into account a careful understanding of the inductive principles involved.
quick and dirty quantum predictions with firmplus. in this paper, we describe our approach to the quantum physics classification problem posed in kdd cup 2004. we used our firmplus recursive partitioning random tree approach to classify the two particle types. we spent almost no time tuning parameters, which made our 3rd place finish somewhat surprising.
randomization in privacy-preserving data mining. suppose there are many clients, each having some personal information, and one server, which is interested only in aggregate, statistically significant, properties of this information. the clients can protect privacy of their data by perturbing it with a randomization algorithm and then submitting the randomized version. the randomization algorithm is chosen so that aggregate properties of the data can be recovered with sufficient precision, while individual entries are significantly distorted. how much distortion is needed to protect privacy can be determined using a privacy measure. several possible privacy measures are known; finding the best measure is an open question. this paper presents some methods and results in randomization for numerical and categorical data, and discusses the issue of measuring privacy.
generalized naive bayes classifiers. this paper presents a generalization of the naive bayes classifier. the method is specifically designed for binary classification problems commonly found in credit scoring and marketing applications. the generalized naive bayes classifier turns out to be a powerful tool for both exploratory and predictive analysis. it can generate accurate predictions through a flexible, non-parametric fitting procedure, while being able to uncover hidden patterns in the data. in this paper, the generalized naive bayes classifier and the original bayes classifier will be demonstrated. also, important ties to logistic regression, the generalized additive model (gam), and weight of evidence will be discussed.
the inference problem: a survey. access control models protect sensitive data from unauthorized disclosure via direct accesses, however, they fail to prevent indirect accesses. indirect data disclosure via inference channels occurs when sensitive information can be inferred from non-sensitive data and metadata. inference channels are often low-bandwidth and complex; nevertheless, detection and removal of inference channels is necessary to guarantee data security. this paper presents a survey of the current and emerging research in data inference control and emphasizes the importance of targeting this so often overlooked problem during database security design.
applying data mining to intrusion detection: the quest for automation, efficiency, and credibility. intrusion detection is an essential component of the layered computer security mechanisms. it requires accurate and efficient models for analyzing a large amount of system and network audit data. this paper is an overview of our research in applying data mining techniques to build intrusion detection models. we describe a framework for mining patterns from system and network audit data, and constructing features according to analysis of intrusion patterns. we discuss approaches for improving the run-time efficiency as well as the credibility of detection models. we report the ideas, algorithms, and prototype systems we have developed, and discuss open research problems.
introduction to the special issue on data mining for health informatics. one of the holy grails of medical research in the next decade is what is often called "personalized" medicine. the goal is to individualize therapy and treatment options, and even prevention strategies. this movement is fueled by the rapid advances made in high-throughput biotechnologies. prime examples include snp (single nucleotide polymorphisms) chips and cgh (comparative genomic hybridization) arrays for dna profiling, oligonucleotide arrays for mrna expression, and advanced mass spectrometry for peptide/protein and metabolite quantitation.
exploiting succinct constraints using fp-trees. since its introduction, frequent-set mining has been generalized to many forms, which include constrained data mining. the use of constraints permits user focus and guidance, enables user exploration and control, and leads to effective pruning of the search space and efficient mining of frequent itemsets. in this paper, we focus on the use of succinct constraints. in particular, we propose a novel algorithm called fps to mine frequent itemsets satisfying succinct constraints. the fps algorithm avoids the generate-and-test paradigm by exploiting succinctness properties of the constraints in a fp-tree based framework. in terms of functionality, our algorithm is capable of handling not just the succinct aggregate constraint, but any succinct constraint in general. moreover, it handles multiple succinct constraints. in terms of performance, our algorithm is more efficient and effective than existing fp-tree based constrained frequent-set mining algorithms.
"in vivo" spam filtering: a challenge problem for kdd. spam, also known as unsolicited commercial email (uce), is the bane of email communication. many data mining researchers have addressed the problem of detecting spam, generally by treating it as a static text classification problem. true in vivo spam filtering has characteristics that make it a rich and challenging domain for data mining. indeed, real-world datasets with these characteristics are typically difficult to acquire and to share. this paper demonstrates some of these characteristics and argues that researchers should pursue in vivo spam filtering as an accessible domain for investigating them.
kdd-99 classifier learning contest: llsoft's results overview. kernel miner is a new data-mining tool based on building the optimal decision forest. the tool won second place in the kdd99 classifier learning contest, august 1999. we describe the kernel miner's approach and method used for solving the contest task. the received results are analyzed and explained.
kdd-cup 2004: protein homology task. in this paper we describe the winning model for the performance measure "lowest ranked homologous sequence" (rkl). this was a subtask of the protein homology prediction task of the kdd cup 2004. the goal was to predict protein homology for different performance metrics. the given data was organized in blocks, each of which corresponds to a specific native sequence. the two metrics average precision (apr) and rkl explicitly make use of this block structure. our solution consists of two parts. the first one is a global classification svm not aware of the block structure. the second part is a k-nearestneighbor scheme for block similarity, used to train ranking svms on the fly. furthermore, we sketch our approach to optimize the root-mean-squared-error and report some alternative solutions that turned out to be suboptimal.
a survey on wavelet applications in data mining. recently there has been significant development in the use of wavelet methods in various data mining processes. however, there has been written no comprehensive survey available on the topic. the goal of this is paper to fill the void. first, the paper presents a high-level data-mining framework that reduces the overall process into smaller components. then applications of wavelets for each component are reviewd. the paper concludes by discussing the impact of wavelets on data mining research and outlining potential future research directions and applications.
distributed higher order association rule mining using information extracted from textual data. the burgconing amount of textual data in distributed sources combined with the obstacles involved in creating and maintaining central repositories motivates the need for effective distributed information extraction and mining techniques. recently, as the need to mine patterns across distributed databases has grown, distributed association rule mining (d-arm) algorithms have been developed. these algorithms, however, assume that the databases are either horizontally or vertically distributed. in the special case of databases populated from information extracted from textual data, existing d-arm algorithms cannot discover rules based on higher-order associations between items in distributed textual documents that are neither vertically nor horizontally distributed, but rather a hybrid of the two. in this article we present d-hotm, a framework for distributed higher order text mining. d-hotm is a hybrid approach that combines information extraction and distributed data mining. we employ a novel information extraction technique to extract meaningful entities from unstructured text in a distributed environment. the information extracted is stored in local databases and a mapping function is applied to identify globally unique keys. based on the extracted information, a novel distributed association rule mining algorithm is applied to discover higher-order associations between items (i.e., entities) in records fragmented across the distributed databases using the keys. unlike existing algorithms, d-hotm requires neither knowledge of a global schema nor that the distribution of data be horizontal or vertical. evaluation methods are proposed to incorporate the performance of the mapping function into the traditional support metric used in arm evaluation. an example application of the algorithm on distributed law enforcement data demonstrates the relevance of d-hotm in the fight against terrorism.
a critical review of multi-objective optimization in data mining: a position paper. this paper addresses the problem of how to evaluate the quality of a model built from the data in a multi-objective optimization scenario, where two or more quality criteria must be simultaneously optimized. a typical example is a scenario where one wants to maximize both the accuracy and the simplicity of a classification model or a candidate attribute subset in attribute selection. one reviews three very different approaches to cope with this problem, namely: (a) transforming the original multi-objective problem into a single-objective problem by using a weighted formula; (b) the lexicographical approach, where the objectives are ranked in order of priority; and (c) the pareto approach, which consists of finding as many non-dominated solutions as possible and returning the set of non-dominated solutions to the user. one also presents a critical review of the case for and against each of these approaches. the general conclusions are that the weighted formula approach -- which is by far the most used in the data mining literature -- is to a large extent an ad-hoc approach for multi-objective optimization, whereas the lexicographic and the pareto approach are more principled approaches, and therefore deserve more attention from the data mining community.
chinese named entity recognition using lexicalized hmms. this paper presents a lexicalized hmm-based approach to chinese named entity recognition (ner). to tackle the problem of unknown words, we unify unknown word identification and ner as a single tagging task on a sequence of known words. to do this, we first employ a known-word bigram-based model to segment a sentence into a sequence of known words, and then apply the uniformly lexicalized hmms to assign each known word a proper hybrid tag that indicates its pattern in forming an entity and the category of the formed entity. our system is able to integrate both the internal formation patterns and the surrounding contextual clues for ner under the framework of hmms. as a result, the performance of the system can be improved without losing its efficiency in training and tagging. we have tested our system using different public corpora. the results show that lexicalized hmms can substantially improve ner performance over standard hmms. the results also indicate that character-based tagging (viz. the tagging based on pure single-character words) is comparable to and can even outperform the relevant known-word based tagging when a lexicalization technique is applied.
kdd cup-2005 report: facing a great challenge. the kdd-cup 2005 competition was held in conjunction with the eleventh acm sigkdd international conference on knowledge discovery and data mining. the task of the kdd-cup 2005 competition was to classify 800,000 internet user search queries into 67 predefined categories. this task is easy to understand, but the lack of straightforward training set, subjective user intents of queries, poor information in short queries, and high noise level make the task very challenge.in this paper, we summarize the competition task, the evaluation method, and the results of the competition. here we only highlight some key techniques used in submitted solutions. the technical details of the solutions from the three award winning teams are available in their papers separately in this issue of sigkdd explorations. at the end, we also share the results of a survey conducted with this year's cup participants. to facilitate research in this area, the task description, data, answer set, and related information of this kdd-cup are published at the kdd-cup 2005 web site: http://www.acm.org/sigs/sigkdd/kdd2005/kddcup.html.
a block-based support vector machine approach to the protein homology prediction task in kdd cup 2004. this paper describes our solution for the protein homology prediction task in kdd cup 2004 competition. this task is modeled as a supervised learning problem with multiple performance metrics. several key characteristics make the problem both novel and challenging, including the concept of data blocks and the presence of large-scale and imbalanced training data. these features make a naive application of the traditional classification algorithms infeasible. our approach focuses on making full use of the abundant information within the blocks, and developing a new technique for reducing and balancing training data to make the support vector machine applicable to this kind of large-scale and imbalanced learning tasks.
extracting statistical data frames from text. we present a framework that bridges the gap between natural language processing (nlp) and text mining. central to this is a new approach to text parameterization that captures many interesting attributes of text usually ignored by standard indices, like the term-document matrix. by storing nlp tags, the new index supports a higher degree of knowledge discovery and pattern finding from text. the index is relatively compact, enabling dynamic search of arbitrary relationships and events in large document collections. we can export search results in formats and data structures that are transparent to statistical analysis tools like s-plusid&reg;. in a number of experiments, we demonstrate how this framework can turn mountains of unstructured information into informative statistical graphs.
classification of heterogeneous gene expression data. recent advanced technologies in dna microarray analysis are intensively applied in disease classification, especially for cancer classification. most recent proposed gene expression classifiers can successfully classify testing samples obtained from the same microarray experiment as training samples with the assumption that the symmetric errors are constant among training and testing samples. however, the classification performance is degraded with heterogeneous testing samples obtained from different microarray experiments. in this paper, we propose the "impact factors" (ifs) to measure the variations between individual classes in training samples and heterogeneous testing samples, and integrate the ifs to classifiers for classification of heterogeneous samples. two publicly available lung adenocarcinomas gene expression data sets are used in our experiments to demonstrate the effectiveness of the ifs. it shows that, with the integration of the ifs to the golub and slonim (gs) and k-nearest neighbors (knn) classifiers, the classifiers can be further improved on the classification accuracy of heterogeneous samples. even more, the classification accuracy of the integrated gs classifier is around 90%.