waashk/dblp huggingface dataset:
on mining general temporal association rules in a publication database. in this paper, we explore a new problem of mining general temporal association rules in publication databases. in essence, a publication database is a set of transactions where each transaction t is a set of items of which each item contains an individual exhibition period. the current model of association rule mining is not able to handle the publication database due to the following fundamental problems, i.e., (1) lack of consideration of the exhibition period of each individual item; (2) lack of an equitable support counting basis for each item. to remedy this, we propose an innovative algorithm progressive-partition-miner (abbreviatedly as ppm) to discover general temporal association rules in a publication database. the basic idea ofppm is to first partition the publication database in light of exhibition periods of items and then progressively accumulate the occurrence count of each candidate 2-itemset based on the intrinsic partitioning characteristics. algorithm ppm is also designed to employ a filtering threshold in each partition to early prune out those cumulatively infrequent 2-itemsets. explicitly, the execution time of ppm is, in orders of magnitude, smaller than those required by the schemes which are directly extended from existing methods.
fs: a random walk based free-form spatial scan statistic for anomalous window detection. often, it is required to identify anomalous windows over a spatial region that reflect unusual rate of occurrence of a specific event of interest. a spatial scan statistic essentially considers a scan window, and identifies anomalous windows by moving the scan window in the region. while spatial scan statistic has been successful, earlier proposals suffer from two limitations: (i) they resrict the scan window to be of a regular shape (e.g., circle, rectangle, cylinder). however, the region of anomaly, in general, is not necessarily of a regular shape. (ii) they take into account autocorrelation among spatial data, but not spatial heterogeneity. as a result, they often result in inaccurate anomalous windows. to address these limitations, we propose a random walk based free-form spatial scan statistic (fs³). application of fs³ on real datasets has shown that it can identify more refined anomalous windows with better likelihood ratio of it being an anomaly, than those identified by earlier spatial scan statistic approaches.
preparations for semantics-based xml mining. xml allows users to define elements using arbitrary words and organize them in a nested structure. these features of xml offer both challenges and opportunities in information retrieval, document management, and data mining. in this paper,we propose a new methodology for preparing xml documents for quantitative determination of similarity between xml documents by taking account of xml semantics (i.e.,meanings of the elements andnested structures of xml documents).accurate quantitative determination of similarity between xml documents provides an important basis for a variety of applications of xml document mining and processing. experiments with xml documents show that ourmethodology provides a 50-100%improvement in determining similarity, over the traditional vector-space model that considers only term-frequency and 100% accuracy in identifying the category of each document from an on-line bookstore.
the computational complexity of high-dimensional correlation search. there is a growing awareness that the popular support metric (often used to guide search in market-basket analysis) is not appropriate for use in every association mining application. support measures only the frequency of co-occurrence of a set of events when determining which pat-terns to report back to the user. it incorporates no rigorous statistical notion of surprise or interest, and many of the patterns deemed interesting by the support metric are uninteresting to the user.however, a positive aspect of support is that search using support is very efficient. the question we address in this paper is: can we retain this efficiency if we move beyond support, and to other, more rigorous metrics? we consider the computational implications of incorporating simple expectation into the data mining task. it turns out that many variations on the problem which incorporate more rigorous tests of dependence (or independence) result in np-hard problem definitions.
dynamic daily-living patterns and association analyses in tele-care systems. tele-care systems aim to carry out intelligent analyses of a person's wellbeing using data about their daily activities. this is a very challenging task because the massive dataset is likely to be erroneous, possibly with misleading sections due to noise or missing values. furthermore, the interpretation of the data is highly sensitive to the lifestyle of the monitored person and the environment in which they interact. in our tele-care project, sensor-network domain knowledge is used to overcome the difficulties of monitoring long-term wellbeing with an imperfect data source. in addition, a fuzzy association analysis is leveraged to implement a dynamic and flexible analysis over individual- and environment-dependent data.
mining minimal distinguishing subsequence patterns with gap constraints. discovering contrasts between collections of data is an important task in data mining. in this paper, we introduce a new type of contrast pattern, called a minimal distinguishing subsequence (mds). an mds is a minimal subsequence that occurs frequently in one class of sequences and infrequently in sequences of another class. it is a natural way of representing strong and succinct contrast information between two sequential datasets and can be useful in applicationssuch as protein comparison, document comparison and building sequential classification models. mining mds patterns is a challenging task and is significantly different from mining contrasts between relational/transactional data. one particularly important type of constraint that can be integrated into the mining process is the gap constraint. we present an efficient algorithm called consgapminer (contrast sequences with gap miner), to mine all mdss satisfying a minimum and maximum gap constraint, plus a maximum length constraint. it employs highly efficient bitset and boolean operations, for powerful gap-based pruning within a prefix growth framework. a performance evaluation with both sparse and dense datasets, demonstrates the scalability of consgapminer and shows its ability to mine patterns from high dimensional datasets at low supports.
an algebra for inductive query evaluation. inductive queries are queries that generate pattern sets.this paper studies properties of boolean inductive queries,i.e. queries that are boolean expressions over monotonicand anti-monotonic constraints. more specifically, we introduceand study algebraic operations on the answer setsof such queries and show how these can be used for constructingand optimizing query plans. special attention isdevoted to the dimension of the queries, i.e. the minimumnumber of version spaces needed to represent the answersets. the framework has been implemented for the patterndomain of strings and experimentally validated.
considering both intra-pattern and inter-pattern anomalies for intrusion detection. various approaches have been proposed to discoverpatterns from system call trails of unix processes tobetter model application behavior. however, thesetechniques only consider relationship between systemcalls (or system audit events). in this paper, we firstrefine the definition of maximal patterns given in [8] andprovide a pattern extraction algorithm to identify suchmaximal patterns. we then add one additional dimensionto the problem domain by also taking into considerationthe overlap relationship between patterns. we argue thatan execution path of an application is usually not anarbitrary combination of various patterns; but rather,they overlap each other in some specific order. suchoverlap relationship characterizes the normal behavior ofthe application. finally, a novel pattern matchingmodule is proposed to detect intrusions based on bothintra-pattern and inter-pattern anomalies. we test thisidea using the data sets obtained from the university ofnew mexico. the experimental results indicate that ourscheme detect significantly more anomalies than thescheme presented in [8] while maintaining a very lowfalse alarm rate.
an optimal linear time algorithm for quasi-monotonic segmentation. monotonicity is a simple yet significant qualitative characteristic. we consider the problem of segmenting an array in up to k segments. we want segments to be as monotonic as possible and to alternate signs. we propose a quality metric for this problem, present an optimal linear time algorithm based on novel formalism, and compare experimentally its performance to a linear time top-down regression algorithm. we show that our algorithm is faster and more accurate. applications include pattern recognition and qualitative modeling.
divide and prosper: comparing models of customer behavior from populations to individuals. this paper compares customer segmentation, 1-to-1, and aggregate marketing approaches across a broad range of experimental settings, including multiple segmentation levels, marketing datasets, dependent variables, and different types of classifiers, segmentation techniques, and predictive measures. our experimental results show that, overall, 1-to-1 modeling significantly outperforms the aggregate approach among high-volume customers and is never worse than aggregate approach among low-volume customers. moreover, the best segmentation techniques tend to outperform 1-to-1 modeling among low-volume customers.
cantree: a tree structure for efficient incremental mining of frequent patterns. since its introduction, frequent-pattern mining has been the subject of numerous studies, including incremental updating. many existing incremental mining algorithms are apriori-based, which are not easily adoptable to fp-tree based frequent-pattern mining. in this paper, we propose a novel tree structure, called cantree (canonical-order tree), that captures the content of the transaction database and orders tree nodes according to some canonical order. by exploiting its nice properties, the cantree can be easily maintained when database transactions are inserted, deleted, and/or modified. for example, the cantree does not require adjustment, merging, and/or splitting of tree nodes during maintenance. no rescan of the entire updated database or reconstruction of a new tree is needed for incremental updating. experimental results show the effectiveness of our cantree.
mining ontological knowledge from domain-specific text documents. traditional text mining systems employ shallow parsing techniques and focus on concept extraction and taxonomic relation extraction. this paper presents a novel system called crctol for mining rich semantic knowledge in the form of ontology from domain-specific text documents. by using a full text parsing technique and incorporating both statistical and lexico-syntactic methods, the knowledge extracted by our system is more concise and contains a richer semantics compared with alternative systems. we conduct a case study wherein crctol extracts ontological knowledge, specifically key concepts and semantic relations, from a terrorism domain text collection. quantitative evaluation, by comparing with a state-of-the-art ontology learning system known as text-to-onto, has shown that crctol produces much better precision and recall for both concept and relation extraction, especially from sentences with complex structures.
mining patterns that respond to actions. data mining focuses on patterns that summarize the data. in this paper, we focus on mining patterns that could change the state by responding to opportunities of actions.
linear causal model discovery using the mml criterion. determining the causal structure of a domain is a keytask in the area of data mining and knowledge discovery.the algorithm proposed by wallace et al. [15] hasdemonstrated its strong ability in discovering linear causalmodels from given data sets. however, some experimentsshowed that this algorithm experienced difficulty in discoveringlinear relations with small deviation, and it occasion-allygives a negative message length, which should not beallowed. in this paper, a more efficient and precise mml encodingscheme is proposed to describe the model structureand the nodes in a linear causal model. the estimation ofdifferent parameters is also derived. empirical results showthat the new algorithm outperformed the previous mml-basedalgorithm in terms of both speed and precision.
learning instance greedily cloning naive bayes for ranking. naive bayes (simply nb) [12] has been widely used in machine learning and data mining as a simple and effective classification algorithm. since its conditional independence assumption is rarely true, researchers have made a substantial amount of effort to improve naive bayes. the related research work can be broadly divided into two approaches: eager learning and lazy learning, depending on when the major computation occurs. different from eager approach, the key idea for extending naive bayes from the lazy approach is to learn a naive bayes for each testing example. in recent years, some lazy extensions of naive bayes have been proposed. for example, snnb [18], lwnb [7], and lbr [19]. all are aiming at improving the classification accuracy of naive bayes. in many real-world machine learning and data mining applications, however, an accurate ranking is more desirable than an accurate classification. responding to this fact, we present a lazy learning algorithm called instance greedily cloning naive bayes (simply igcnb) in this paper. our motivation is to improve naive bayes' ranking performance measured by auc [4, 14]. we experimentally tested our algorithm, using the whole 36 uci datasets recommended by weka [1], and compared it to c4.4 [16], nb [12], snnb [18] and lwnb [7]. the experimental results show that our algorithm outperforms all the other algorithms used to compare significantly in yielding accurate ranking.
mining production data with neural network & cart. this paper presents the preliminary results of a datamining study of a production line involving hundreds ofvariables related to mechanical, chemical, electrical andmagnetic processes involved in manufacturing coatedglass. the study was performed using two nonlinear,nonparametric approaches, namely neural network andcart, to model the relationship between the qualities ofthe coating and machine readings. furthermore, neuralnetwork sensitivity analysis and cart variable rankingswere used to gain insight into the coating process. ourinitial results show the promise of data mining techniquesto improve the production.
cmar: accurate and efficient classification based on multiple class-association rules. previous studies propose that associative classification has high classification accuracy and strong flexibility at handling unstructured data. however, it still suffers from the huge set of mined rules and sometimes biased classification or overfitting since the classificationis based on only single high-confidence rule. in this study, we propose new associative classification method, cmar, i.e., classification based on multiple association rules. the method extends an efficient frequent pattern mining method, fp-growth ,constructs classdistribution-associated fp-tree, and mines large database efficiently. moreover, it applies cr-tree structure to store and retrieve mined association rulesefficiently, and prunes rules effectively based on confidence, correlation and database coverage. the classification is performed based on weighted x2 analysis using multiple strong association rules. our extensive experiments on 26 databases from uci machine learning database repository show that cmar is consistent, highly effective at classificationof various kinds of databases and has better average classificationaccuracy in comparison with cba and c4.5.moreover,our performancestudy shows that the method is highly efficient and scalable in comparison with other reported associative classification methods.
classifying biomedical citations without labeled training examples. in this paper we introduce a novel technique for classifying text citations without labeled training examples. we first utilize the search results of a general search engine as original training data. we then proposed a mutually reinforcing learning algorithm (mrl) to mine the classification knowledge and to "clean" the training data. with the help of a set of established domain-specific ontological terms or keywords, the mrl mining step derives the relevant classification knowledge. the mrl cleaning step then builds a naive bayes classifier based on the mined classification knowledge and tries to clean the training set. the mrl algorithm is iteratively applied until a clean training set is obtained. we show the effectiveness of the proposed technique in the classification of biomedical citations from a large medical literature database.
mining image features for efficient query processing. the number of feature required to depict an image can be very large. using all features simultaneously to measure image similarity and to learn image query-concepts can suffer from the problem of dimensionality curse ,which degrades both search accuracy and search peed. regarding search accuracy, the presence of irrelevant features with respect to a query can contaminate similarity measurement, and hence decrease both the recall and precision of thatquery. to remedy this problem, we present a mining method that learns online user query concept and identities important features quickly. regarding search speed, the presence of a large number of feature can low down query-concept learning and indexing performance. we propose a divide-and-conquer method that divides the concept-learning task into g subtasks to achieve speedup. we notice that a task must be divided carefully, or search accuracy maysuffer. we thus propose a genetic-based mining algorithm to discover good feature groupings. through analysis and mining result, we observe that organizing image features in a multi-resolution manner, and minimizing intra-group feature correlation, can peed up query-concept learning substantially while maintaining high search accuracy.
improving medical/biological data classification performance by wavelet preprocessing. many real-world datasets contain noise and noisecould degrade the performances of learning algorithms.motivated from the success of wavelet denoisingtechniques in image data, we explore a generalsolution to alleviate the effect of noisy databy wavelet preprocessing for medical/biological dataclassification. our experiments are divided into twocategories: one is of different classification algorithmson a specific database (ecoli [6]) and the other isof a specific classification algorithm (decision tree)on different databases. the experiment results showthat the wavelet denoising of noisy data is able to improvethe accuracies of those classification methods,if the localities of the attributes are strong enough.
adaptive and resource-aware mining of frequent sets. the performance of an algorithm that mines frequent sets from transactional databases may severely depend on the specific features of the data being analyzed. moreover, some architectural characteristics of the computational platform used - e.g. the available main memory - can dramatically change its runtime behavior. in this paper we present dci (direct count & intersect), an efficient algorithm for discovering frequent sets from large databases. due to the multiple heuristics strategies adopted, dci can adapt its behavior not only to the features of the specific computing platform, but also to the features of the datasetbeing mined, so that it results very effective in mining both short and long patterns from sparse and dense datasets. finally we also discuss the parallelization strategies adopted in the design of pardci, a distributed and multi-threaded implementation of dci.
evolutionary structure learning algorithm for bayesian network and penalized mutual information metric. this paper formulates the problem of learning bayesian network structures from data as determining the structure that best approximates the probability distribution indicated by the data. a new metric, penalized mutual information metric, is proposed, and a evolutionary algorithm is designed to search for the best structure among alternatives. the experimental results show that this approach is reliable and promising.
solving the fragmentation problem of decision trees by discovering boundary emerging patterns. the single coverage constraint discourages a decisiontree to contain many significant rules. the loss of significantrules leads to a loss in accuracy. on the other hand, thefragmentation problem causes a decision tree to contain toomany minor rules. the presence of minor rules decreasesaccuracy. we propose to use emerging patterns to solvethese problems. in our approach, many globally significantrules can be discovered. extensive experimental results ongene expression datasets show that our approach are moreaccurate than single c4.5 trees, and are also better thanbagged or boosted c4.5 trees.
spatial interest pixels (sips): useful low-level features of visual media data. visual media data such as an image is the raw data representationfor many important applications. the biggestchallenge in using visual media data comes from the extremelyhigh dimensionality. we present a comparativestudy on spatial interest pixels (sips), including eight-way(a novel sip miner), harris, and lucas-kanade, whose extractionis considered as an important step in reducing thedimensionality of visual media data. with extensive casestudies, we have shown the usefulness of sips as the low-levelfeatures of visual media data. a class-preserving dimensionreduction algorithm (using gsvd) is applied tofurther reduce the dimension of feature vectors based onsips. the experiments showed its superiority over pca.
multi-stage classification. while much research has focused on methods for evaluating and maximizing the accuracy of classifiers either individually or in ensembles, little effort has been devoted to analyzing how classifiers are typically deployed in practice. in many domains, classifiers are used as part of a multi-stage process that increases accuracy at the expense of more data collection and/or more processing resources as the likelihood of a positive class label increases. this paper systematically explores the tradeoffs inherent in constructing these multi-stage classifiers from a series of increasingly accurate and expensive individual classifiers, considering a variety of metrics such as accuracy, cost/benefit ratio, and lift. it suggests architectures appropriate for both independent instances and for highly linked data.
direct interesting rule generation. an association rule generation algorithm usually generatestoo many rules including a lot of uninteresting ones.many interestingness criteria are proposed to prune thoseuninteresting rules. however, they work in post-pruningprocess and hence do not improve the rule generation ef£ciency. in this paper, we discuss properties of informativerule set and conclude that the informative rule set includesall interesting rules measured by many commonly used interestingnesscriteria, and that rules excluded by the informativerule set are forwardly prunable, i.e. they can be removedin the rule generation process instead of post pruning.based on these properties, we propose a direct interestingrule generation algorithm, dig, to directly generateinteresting rules de£ned by any of 12 interestingness criteriadiscussed in this paper. we further show experimentallythat dig is faster and uses less memory than apriori.