waashk/dblp huggingface dataset:
wifisviz: effective visualization of frequent itemsets. frequent itemset mining plays an essential role in the mining of many different patterns. most existing frequent itemset mining algorithms return the mined results--namely, frequent itemsets--in the form of textual lists. however, the use of visual representation can enhance the user understanding of the inherent relations in a collection of frequent itemsets. in this paper, we propose an effective visualizer, called wifisviz, to display the mined frequent itemsets. wifisviz provides users with an overview and details about the itemsets. moreover, this visualizer is also equipped with several interactive features for effective visualization of the frequent itemsets mined from various real-life applications.
spatiotemporal relational probability trees: an introduction. we introduce spatiotemporal relational probability trees (srpts), probability estimation trees for relational data that can vary in both space and time. the srpt algorithm addresses the exponential increase in search complexity through sampling. we validate the srpt using a simulated data set and we empirically demonstrate the srpt algorithm on two real-world data sets.
fast and memory efficient mining of high utility itemsets in data streams. efficient mining of high utility itemsets has become one of the most interesting data mining tasks with broad applications. in this paper, we proposed two efficient one-pass algorithms, mhui-bit and mhui-tid, for mining high utility itemsets from data streams within a transaction-sensitive sliding window. two effective representations of item information and an extended lexicographical tree-based summary data structure are developed to improve the efficiency of mining high utility itemsets. experimental results show that the proposed algorithms outperform than the existing algorithms for mining high utility itemsets from data streams.
m3miml: a maximum margin method for multi-instance multi-label learning. multi-instance multi-label learning (miml) deals with the problem where each training example is associated with not only multiple instances but also multiple class labels. previous miml algorithms work by identifying its equivalence in degenerated versions of multi-instance multi-label learning. however, useful information encoded in training examples may get lost during the identification process. in this paper, a maximum margin method is proposed for miml which directly exploits the connections between instances and labels. the learning task is formulated as a quadratic programming (qp) problem and implemented in its dual form. applications to scene classification and text categorization show that the proposed approach achieves superior performance over existing miml methods.
learning the latent semantic space for ranking in text retrieval. subspace learning techniques for text analysis, such as latent semantic indexing (lsi), have been widely studied in the past decade. however, to our best knowledge, no previous study has leveraged the rank information for subspace learning in ranking tasks. in this paper, we propose a novel algorithm, called learning latent semantics for ranking (llsr), to seek the optimal latent semantic space tailored to the ranking tasks. we first present a dual explanation for the classical latent semantic indexing (lsi) algorithm, namely learning the so-called latent semantic space (lss) to encode the data information. then, to handle the increasing amount of training data for the practical ranking tasks, we propose a novel objective function to derive the optimal lss for ranking. experimental results on two smart sub-collections and a trec dataset show that llsr effectively improves the ranking performance compared with the classical lsi algorithm and ranking without subspace learning.
fast counting of triangles in large real networks without counting: algorithms and laws. triangles are important for real world social networks, lying at the heart of the clustering coefficient and of the transitivity ratio. however, straight-forward and even approximate counting algorithms can be slow, trying to execute or approximate the equivalent of a 3-way database join. in this paper, we provide two algorithms, the eigen triangle for counting the total number of triangles in a graph, and the eigen triangle local algorithm that gives the count of triangles that contain a desired node. additional contributions include the following:(a) we show that both algorithms achieve excellent accuracy, with up to ~1000x faster execution time, on several, real graphs and (b) we discover two new power laws (degree-triangle and triangle participation laws) with surprising properties.
quantitative association analysis using tree hierarchies. association analysis arises in many important applications such as bioinformatics and business intelligence. given a large collection of measurements over a set of samples, association analysis aims to find dependencies of target variables to subsets of measurements. most previous algorithms adopt a two-stage approach; they first group samples based on the similarity in the subset of measurements, and then they examine the association between these groups and the specified target variables without considering the inter-group similarities or alternative groupings. this can lead to cases where the strength of association depends significantly on arbitrary clustering choices. in this paper, we propose a tree-based method for quantitative association analysis. tree hierarchies derived from sample similarities represent many possible sample groupings. they also provide a natural way to incorporate domain knowledge such as ontologies and to identify and remove outliers. given a tree hierarchy, our association analysis evaluates all possible groupings and selects the one with strongest association to the target variable. we introduce an efficient algorithm, treeqa, to systematically explore the search-space of all possible groupings in a set of input trees, with integrated permutation tests. experimental results show that treeqa is able to handlelarge-scale association analysis very efficiently and is more effective and robust in association analysis than previous methods.
formal models for expert finding on dblp bibliography data. finding relevant experts in a specific field is often crucial for consulting, both in industry and in academia. the aim of this paper is to address the expert-finding task in a real world academic field. we present three models for expert finding based on the large-scale dblp bibliography and google scholar for data supplementation. the first, a novel weighted language model, models an expert candidate based on the relevance and importance of associated documents by introducing a document prior probability, and achieves much better results than the basic language model. the second, a topic-based model, represents each candidate as a weighted sum of multiple topics, whilst the third, a hybrid model, combines the language model and the topic-based model. we evaluate our system using a benchmark dataset based on human relevance judgments of how well the expertise of proposed experts matches a query topic. evaluation results show that our hybrid model outperforms other models in nearly all metrics.
finding good itemsets by packing data. the problem of selecting small groups of itemsets that represent the data well has recently gained a lot of attention. we approach the problem by searching for the itemsets that compress the data efficiently. as a compression technique we use decision trees combined with a refined version of mdl. more formally, assuming that the items are ordered, we create a decision tree for each item that may only depend on the previous items. our approach allows us to find complex interactions between the attributes, not just co-occurrences of 1s. further, we present a link between the itemsets and the decision trees and use this link to export the itemsets from the decision trees. in this paper we present two algorithms. the first one is a simple greedy approach that builds a family of itemsets directly from data. the second one, given a collection of candidate itemsets, selects a small subset of these itemsets. our experiments show that these approaches result in compact and high quality descriptions of the data.
toward faster nonnegative matrix factorization: a new algorithm and comparisons. nonnegative matrix factorization (nmf) is a dimension reduction method that has been widely used for various tasks including text mining, pattern analysis, clustering, and cancer class discovery. the mathematical formulation for nmf appears as a non-convex optimization problem, and various types of algorithms have been devised to solve the problem. the alternating nonnegative least squares (anls) framework is a block coordinate descent approach for solving nmf, which was recently shown to be theoretically sound and empirically efficient. in this paper, we present a novel algorithm for nmf based on the anls framework. our new algorithm builds upon the block principal pivoting method for the nonnegativity constrained least squares problem that overcomes some limitations of active set methods. we introduce ideas to efficiently extend the block principal pivoting method within the context of nmf computation. our algorithm inherits the convergence theory of the anls framework and can easily be extended to other constrained nmf formulations. comparisons of algorithms using datasets that are from real life applications as well as those artificially generated show that the proposed new algorithm outperforms existing ones in computational speed.
a joint matrix factorization approach to unsupervised action categorization. in this paper, a novel unsupervised approach to mining categories from action video sequences is presented. this approach consists of two modules: action representation and learning model. videos are regarded as spatially distributed dynamic pixel time series, which are quantized into pixel prototypes. after replacing the pixel time eries with their corresponding prototype labels, the video sequences are compressed into 2d action matrices. we put these matrices together to form an multi-action tensor, and propose the joint matrix factorization method to simultaneously cluster the pixel prototypes into pixel signatures, and matrices into action classes. the approach is tested on public and popular weizmann data set, and promising results are achieved.
a fast method to mine frequent subsequences from graph sequence data. in recent years, the mining of a complete set of frequent subgraphs from labeled graph data has been extensively studied.however, to our best knowledge, almost no methods have been proposed to find frequent subsequences of graphs from a set of graph sequences. in this paper, we define a novel class of graph subsequences by introducing axiomatic rules of graph transformation, their admissibility constraints and a union graph. then we propose an efficient approach named "gtrace'' to enumerate frequent transformation subsequences (ftss) of graphs from a given set of graph sequences. its fundamental performance has been evaluated by using artificial datasets, and its practicality has been confirmed through the experiments using real world datasets.
a recommendation system for preconditioned iterative solvers. preconditioned iterative methods are often used to solve very large sparse systems of linear systems that arise in many scientific and engineering applications. the performance and robustness of these solvers is extremely sensitive to the choice of multiple preconditioner and solver parameters. users of iterative methods often encounter an overwhelming number of combinations of choices for solvers, matrix preprocessing steps, preconditioners, and their parameters. the lack of a unified theoretical analysis of preconditioners coupled with limited knowledge of their interaction with linear systems makes it highly challenging for practitioners to choose good solver configurations. in this paper, we propose a novel, multi-stage learning based methodology for determining the best solver configurations to optimize the desired performance behavior for any given linear system. empirical results over real performance data for the hypre iterative solver package demonstrate the efficacy and flexibility of the proposed approach.
mining order-preserving submatrices from data with repeated measurements. order-preserving submatrices (opsm's) have been shown useful in capturing concurrent patterns in data when the relative magnitudes of data items are more important than their absolute values. to cope with data noise, repeated experiments are often conducted to collect multiple measurements. we propose and study a more robust version of opsm, where each data item is represented by a set of values obtained from replicated experiments. we call the new problem opsm-rm (opsm with repeated measurements). we define opsm-rm based on a number of practical requirements. we discuss the computational challenges of opsm-rm and propose a generic mining algorithm. we further propose a series of techniques to speed up two time-dominating components of the algorithm. we clearly show the effectiveness of our methods through a series of experiments conducted on real microarray data.
predicting future decision trees from evolving data. recognizing and analyzing change is an important human virtue because it enables us to anticipate future scenarios and thus allows us to act pro-actively. one approach to understand change within a domain is to analyze how modelsand patterns evolve. knowing how a model changes over time is suggesting to ask: can we use this knowledge to learn a model in anticipation, such that it better reflects the near-future characteristics of an evolving domain? in this paper we provide an answer to this question by presenting an algorithm which predicts future decision trees based ona model of change. in particular, this algorithm encompasses a novel approach to change mining which is based on analyzing the changes of the decisions made during model learning. the proposed approach can also be applied to other types of classifiers and thus provides a basis for future research. we present our first experimental results which show that anticipated decision trees have the potential to outperform trees learned on the most recent data.
clustering distributed time series in sensor networks. event detection is a critical task in sensor networks, especially for environmental monitoring applications. traditional solutions to event detection are based on analyzing one-shot data points, which might incur a high false alarm rate because sensor data is inherently unreliable and noisy. to address this issue, we proposea novel distributed single-pass incremental clustering (dsic) technique to cluster the time series obtained at sensor nodes based on their underlying trends. in order to achieve scalability and energy-efficiency, our dsic technique uses a hierarchical structure of sensor networks as the underlying infrastructure. the algorithm first compresses the time series produced at individual sensor nodes into a compact representation using haar wavelettransform, and then, based on dynamic time warping distances, hierarchically groups the approximate time series into a global clustering model in an incremental manner. experimental results on both real data and synthetic data demonstrate that our dsic algorithm is accurate, energy-efficient and robust with respect tonetwork topology changes.
tofa: trace oriented feature analysis in text categorization. dimension reduction for large-scale text data is attracting much attention lately due to the rapid growth of world wide web. we can consider dimension reduction algorithms in two categories: feature extraction and feature selection. an important problem remains: it has been difficult to integrate these two algorithm categories into a single framework, making it difficult to reap the benefit of both. in this paper, we formulate the two algorithm categories through a unified optimization framework. under this framework, we develop a novel feature selection algorithm called trace oriented feature analysis (tofa). the novel objective function of tofa is a unified framework that integrates many prominent feature extraction algorithms such as unsupervised principal component analysis and supervised maximum margin criterion are special cases of it. thus tofa can process not only supervised problem but also unsupervised and semi-supervised problems. experimental results on real text datasets demonstrate the effectiveness and efficiency of tofa.
web mining for understanding stories through graph visualisation. rich information spaces (like the web or scientific publications) are full of "stories": sets of statements that evolve over time, manifested as, for example, collections of newspaper articles reporting events relating to an evolving crime investigation, sets of news articles and blog posts accompanying the development of a political election campaign, or sequences of scientific papers on a topic. in this paper, we propose a method and a visualisation tool for mapping and interacting with such stories. in contrast to existing approaches, our method concentrates on relational information and on local patterns rather than on the occurrence of individual concepts and global models. in addition, we present an evaluation framework. a real-life case study is used to illustrate and evaluate the method and tool.
multiplicative mixture models for overlapping clustering. the problem of overlapping clustering, where a point is allowed to belong to multiple clusters, is becoming increasingly important in a variety of applications. in this paper, we present an overlapping clustering algorithm based on multiplicative mixture models. we analyze a general setting where each component of the multiplicative mixture is from an exponential family, and present an efficient alternating maximization algorithm to learn the model and infer overlapping clusters. we also show that when each component is assumed to be a gaussian, we can apply the kernel trick leading to non-linear cluster separators and obtain better clustering quality. the efficacy of the proposed algorithms is demonstrated usingexperiments on both uci benchmark datasets and a microarray gene expression dataset.
anomaly detection support vector machine and its application to fault diagnosis. we address the issue of classification problems in the following situation: test data include data belonging to unlearned classes. to address this issue, most previous works have taken two-stage strategies where unclear data are detected using an anomaly detection algorithm in the first stage while the rest of data are classified into learned classes using a classification algorithm in the second stage. in this study, we propose anomaly detection support vector machine (adsvm) which unifies classification and anomaly detection. adsvm is unique in comparison with the previous work in that it addresses the two problems simultaneously. we also propose a multiclass extension of adsvm that uses a pairwise voting strategy. we empirically present that adsvm outperforms two-stage algorithms in application to an real automobile fault dataset, as well as to uci benchmark datasets.
hirel: an incremental clustering algorithm for relational datasets. traditional clustering approaches usually analyze static datasets in which objects are kept unchanged after being processed, but many practical datasets are dynamically modified which means some previously learned patterns have to be updated accordingly. re-clustering the whole dataset from scratch is not a good choice due to the frequent data modifications and the limited out-of-service time, so the development of incremental clustering approaches is highly desirable. besides that, propositional clustering algorithms are not suitable for relational datasets because of their quadratic computational complexity. in this paper, we propose an incremental clustering algorithm that requires only one pass of the relational dataset. the utilization of the representative objects and the balanced search tree greatly accelerate the learning procedure. experimental results prove the effectiveness of our algorithm.
graph-based iterative hybrid feature selection. when the number of labeled examples is limited, traditional supervised feature selection techniques often fail due to sample selection bias or unrepresentative sample problem. to solve this, semi-supervised feature selection techniques exploit the statistical information of both labeled and unlabeled examples in the same time. however, the results of semi-supervised feature selection can be at times unsatisfactory, and the culprit is on how to effectively use the unlabeled data. quite different from both supervised and semi-supervised feature selection, we propose a “hybrid”framework based on graph models. we first apply supervised methods to select a small set of most critical features from the labeled data. importantly, these initial features might otherwise be missed when selection is performed onthe labeled and unlabeled examples simultaneously. next,this initial feature set is expanded and corrected with the use of unlabeled data. we formally analyze why the expected performance of the hybrid framework is better than both supervised and semi-supervised feature selection. experimental results demonstrate that the proposed method outperforms both traditional supervised and state-of-the-art semisupervised feature selection algorithms by at least 10% inaccuracy on a number of text and biomedical problems with thousands of features to choose from. software and dataset is available from the authors.
publishing sensitive transactions for itemset utility. we consider the problem of publishing sensitive transaction data with privacy preservation. high dimensionality of transaction data poses unique challenges on data privacy and data utility. on one hand, re-identification attacks tend to use a subset of items that infrequently occur in transactions, called moles. on the other hand, data mining applications typically depend on subsets of items that frequently occur in transactions, called nuggets. thus the problem is how to eliminate all moles while retaining nuggets as much as possible. a challenge is that moles and nuggets are multi-dimensional with exponential growth and are tangled together by shared items. we present a novel and scalable solution to this problem. the novelty lies in a compact border data structure that eliminates the need of generating all moles and nuggets.
using wikipedia for co-clustering based cross-domain text classification. traditional approaches to document classification requires labeled data in order to construct reliable and accurate classifiers. unfortunately, labeled data are seldom available, and often too expensive to obtain. given a learning task for which training data are not available, abundant labeled data may exist for a different but related domain. one would like to use the related labeled data as auxiliary information to accomplish the classification task in the target domain. recently, the paradigm of transfer learning has been introduced to enable effective learning strategies when auxiliary data obey a different probability distribution. a co-clustering based classification algorithm has been previously proposed to tackle cross-domain text classification. in this work, we extend the idea underlying this approach by making the latent semantic relationship between the two domains explicit. this goal is achieved with the use of wikipedia. as a result, the pathway that allows to propagate labels between the two domains not only captures common words, but also semantic concepts based on the content of documents. we empirically demonstrate the efficacy of our semantic-based approach to cross-domain classification using a variety of real data.