waashk/dblp huggingface dataset:
social action tracking via noise tolerant time-varying factor graphs. it is well known that users' behaviors (actions) in a social network are influenced by various factors such as personal interests, social influence, and global trends. however, few publications systematically study how social actions evolve in a dynamic social network and to what extent different factors affect the user actions. in this paper, we propose a noise tolerant time-varying factor graph model (ntt-fgm) for modeling and predicting social actions. ntt-fgm simultaneously models social network structure, user attributes and user action history for better prediction of the users' future actions. more specifically, a user's action at time t is generated by her latent state at t, which is influenced by her attributes, her own latent state at time t-1 and her neighbors' states at time t and t-1. based on this intuition, we formalize the social action tracking problem using the ntt-fgm model; then present an efficient algorithm to learn the model, by combining the ideas from both continuous linear system and markov random field. finally, we present a case study of our model on predicting future social actions. we validate the model on three different types of real-world data sets. qualitatively, our model can uncover some interesting patterns of the social dynamics. quantitatively, experimental results show that the proposed method outperforms several baseline methods for action prediction.
neighbor query friendly compression of social networks. compressing social networks can substantially facilitate mining and advanced analysis of large social networks. preferably, social networks should be compressed in a way that they still can be queried efficiently without decompression. arguably, neighbor queries, which search for all neighbors of a query vertex, are the most essential operations on social networks. can we compress social networks effectively in a neighbor query friendly manner, that is, neighbor queries still can be answered in sublinear time using the compression? in this paper, we develop an effective social network compression approach achieved by a novel eulerian data structure using multi-position linearizations of directed graphs. our method comes with a nontrivial theoretical bound on the compression rate. to the best of our knowledge, our approach is the first that can answer both out-neighbor and in-neighbor queries in sublinear time. an extensive empirical study on more than a dozen benchmark real data sets verifies our design.
cold start link prediction. in the traditional link prediction problem, a snapshot of a social network is used as a starting point to predict, by means of graph-theoretic measures, the links that are likely to appear in the future. in this paper, we introduce cold start link prediction as the problem of predicting the structure of a social network when the network itself is totally missing while some other information regarding the nodes is available. we propose a two-phase method based on the bootstrap probabilistic graph. the first phase generates an implicit social network under the form of a probabilistic graph. the second phase applies probabilistic graph-based measures to produce the final prediction. we assess our method empirically over a large data collection obtained from flickr, using interest groups as the initial information. the experiments confirm the effectiveness of our approach.
direct mining of discriminative patterns for classifying uncertain data. classification is one of the most essential tasks in data mining. unlike other methods, associative classification tries to find all the frequent patterns existing in the input categorical data satisfying a user-specified minimum support and/or other discrimination measures like minimum confidence or information-gain. those patterns are used later either as rules for rule-based classifier or training features for support vector machine (svm) classifier, after a feature selection procedure which usually tries to cover as many as the input instances with the most discriminative patterns in various manners. several algorithms have also been proposed to mine the most discriminative patterns directly without costly feature selection. previous empirical results show that associative classification could provide better classification accuracy over many datasets. recently, many studies have been conducted on uncertain data, where fields of uncertain attributes no longer have certain values. instead probability distribution functions are adopted to represent the possible values and their corresponding probabilities. the uncertainty is usually caused by noise, measurement limits, or other possible factors. several algorithms have been proposed to solve the classification problem on uncertain data recently, for example by extending traditional rule-based classifier and decision tree to work on uncertain data. in this paper, we propose a novel algorithm uharmony which mines discriminative patterns directly and effectively from uncertain data as classification features/rules, to help train either svm or rule-based classifier. since patterns are discovered directly from the input database, feature selection usually taking a great amount of time could be avoided completely. effective method for computation of expected confidence of the mined patterns used as the measurement of discrimination is also proposed. empirical results show that using svm classifier our algorithm uharmony outperforms the state-of-the-art uncertain data classification algorithms significantly with 4% to 10% improvements on average in accuracy on 30 categorical datasets under varying uncertain degree and uncertain attribute number.
a probabilistic model for personalized tag prediction. social tagging systems have become increasingly popular for sharing and organizing web resources. tag prediction is a common feature of social tagging systems. social tagging by nature is an incremental process, meaning that once a user has saved a web page with tags, the tagging system can provide more accurate predictions for the user, based on user's incremental behaviors. however, existing tag prediction methods do not consider this important factor, in which their training and test datasets are either split by a fixed time stamp or randomly sampled from a larger corpus. in our temporal experiments, we perform a time-sensitive sampling on an existing public dataset, resulting in a new scenario which is much closer to "real-world". in this paper, we address the problem of tag prediction by proposing a probabilistic model for personalized tag prediction. the model is a bayesian approach, and integrates three factors - ego-centric effect, environmental effects and web page content. two methods - both intuitive calculation and learning optimization - are provided for parameter estimation. pure graphbased methods which may have significant constraints (such as every user, every item and every tag has to occur in at least p posts), cannot make a prediction in most of "real world" cases while our model improves the f-measure by over 30% compared to a leading algorithm, in our "real-world" use case.
the community-search problem and how to plan a successful cocktail party. a lot of research in graph mining has been devoted in the discovery of communities. most of the work has focused in the scenario where communities need to be discovered with only reference to the input graph. however, for many interesting applications one is interested in finding the community formed by a given set of nodes. in this paper we study a query-dependent variant of the community-detection problem, which we call the community-search problem: given a graph g, and a set of query nodes in the graph, we seek to find a subgraph of g that contains the query nodes and it is densely connected. we motivate a measure of density based on minimum degree and distance constraints, and we develop an optimum greedy algorithm for this measure. we proceed by characterizing a class of monotone constraints and we generalize our algorithm to compute optimum solutions satisfying any set of monotone constraints. finally we modify the greedy algorithm and we present two heuristic algorithms that find communities of size no greater than a specified upper bound. our experimental evaluation on real datasets demonstrates the efficiency of the proposed algorithms and the quality of the solutions we obtain.
pet: a statistical model for popular events tracking in social communities. user generated information in online communities has been characterized with the mixture of a text stream and a network structure both changing over time. a good example is a web-blogging community with the daily blog posts and a social network of bloggers. an important task of analyzing an online community is to observe and track the popular events, or topics that evolve over time in the community. existing approaches usually focus on either the burstiness of topics or the evolution of networks, but ignoring the interplay between textual topics and network structures. in this paper, we formally define the problem of popular event tracking in online communities (pet), focusing on the interplay between texts and networks. we propose a novel statistical method that models the the popularity of events over time, taking into consideration the burstiness of user interest, information diffusion on the network structure, and the evolution of textual topics. specifically, a gibbs random field is defined to model the influence of historic status and the dependency relationships in the graph; thereafter a topic model generates the words in text content of the event, regularized by the gibbs random field. we prove that two classic models in information diffusion and text burstiness are special cases of our model under certain situations. empirical experiments with two different communities and datasets (i.e., twitter and dblp) show that our approach is effective and outperforms existing approaches.
evaluating online ad campaigns in a pipeline: causal models at scale. display ads proliferate on the web, but are they effective? or are they irrelevant in light of all the other advertising that people see? we describe a way to answer these questions, quickly and accurately, without randomized experiments, surveys, focus groups or expert data analysts. doubly robust estimation protects against the selection bias that is inherent in observational data, and a nonparametric test that is based on irrelevant outcomes provides further defense. simulations based on realistic scenarios show that the resulting estimates are more robust to selection bias than traditional alternatives, such as regression modeling or propensity scoring. moreover, computations are fast enough that all processing, from data retrieval through estimation, testing, validation and report generation, proceeds in an automated pipeline, without anyone needing to see the raw data.
multi-task learning for boosting with application to web search ranking. in this paper we propose a novel algorithm for multi-task learning with boosted decision trees. we learn several different learning tasks with a joint model, explicitly addressing the specifics of each learning task with task-specific parameters and the commonalities between them through shared parameters. this enables implicit data sharing and regularization. we evaluate our learning method on web-search ranking data sets from several countries. here, multitask learning is particularly helpful as data sets from different countries vary largely in size because of the cost of editorial judgments. our experiments validate that learning various tasks jointly can lead to significant improvements in performance with surprising reliability.
combining predictions for accurate recommender systems. we analyze the application of ensemble learning to recommender systems on the netflix prize dataset. for our analysis we use a set of diverse state-of-the-art collaborative filtering (cf) algorithms, which include: svd, neighborhood based approaches, restricted boltzmann machine, asymmetric factor model and global effects. we show that linearly combining (blending) a set of cf algorithms increases the accuracy and outperforms any single cf algorithm. furthermore, we show how to use ensemble methods for blending predictors in order to outperform a single blending algorithm. the dataset and the source code for the ensemble blending are available online.
an integrated machine learning approach to stroke prediction. stroke is the third leading cause of death and the principal cause of serious long-term disability in the united states. accurate prediction of stroke is highly valuable for early intervention and treatment. in this study, we compare the cox proportional hazards model with a machine learning approach for stroke prediction on the cardiovascular health study (chs) dataset. specifically, we consider the common problems of data imputation, feature selection, and prediction in medical datasets. we propose a novel automatic feature selection algorithm that selects robust features based on our proposed heuristic: conservative mean. combined with support vector machines (svms), our proposed feature selection algorithm achieves a greater area under the roc curve (auc) as compared to the cox proportional hazards model and l1 regularized cox feature selection algorithm. furthermore, we present a margin-based censored regression algorithm that combines the concept of margin-based classifiers with censored regression to achieve a better concordance index than the cox model. overall, our approach outperforms the current state-of-the-art in both metrics of auc and concordance index. in addition, our work has also identified potential risk factors that have not been discovered by traditional approaches. our method can be applied to clinical prediction of other diseases, where missing data are common and risk factors are not well understood.
optimizing debt collections using constrained reinforcement learning. the problem of optimally managing the collections process by taxation authorities is one of prime importance, not only for the revenue it brings but also as a means to administer a fair taxing system. the analogous problem of debt collections management in the private sector, such as banks and credit card companies, is also increasingly gaining attention. with the recent successes in the applications of data analytics and optimization to various business areas, the question arises to what extent such collections processes can be improved by use of leading edge data modeling and optimization techniques. in this paper, we propose and develop a novel approach to this problem based on the framework of constrained markov decision process (mdp), and report on our experience in an actual deployment of a tax collections optimization system at new york state department of taxation and finance (nys dtf).
discovering frequent subgraphs over uncertain graph databases under probabilistic semantics. frequent subgraph mining has been extensively studied on certain graph data. however, uncertainties are inherently accompanied with graph data in practice, and there is very few work on mining uncertain graph data. this paper investigates frequent subgraph mining on uncertain graphs under probabilistic semantics. specifically, a measure called φ-frequent probability is introduced to evaluate the degree of recurrence of subgraphs. given a set of uncertain graphs and two numbers 0 s with probability at least (1 - δ/2)s, where s is the number of edges of s. in addition, it is thoroughly discussed how to set δ to guarantee the overall approximation quality of the algorithm. the extensive experiments on real uncertain graph data verify that the algorithm is efficient and that the mining results have very high quality.
minefleet®: an overview of a widely adopted distributed vehicle performance data mining system. this paper describes the minefleet distributed vehicle performance data mining system designed for commercial fleets. minefleet analyzes high throughput data streams onboard the vehicle, generates the analytics, sends those to the remote server over the wide-area wireless networks and offers them to the fleet managers using stand-alone and web-based user-interface. the paper describes the overall architecture of the system, business needs, and shares experience from successful large-scale commercial deployments. minefleet is probably one of the first commercially successful distributed data stream mining systems. this patented technology has been adopted, productized, and commercially offered by many large companies in the mobile resource management and gps fleet tracking industry. this paper offers an overview of the system and offers a detailed analysis of what made it work.
k-support anonymity based on pseudo taxonomy for outsourcing of frequent itemset mining. for any outsourcing service, privacy is a major concern. this paper focuses on outsourcing frequent itemset mining and examines the issue on how to protect privacy against the case where the attackers have precise knowledge on the supports of some items. we propose a new approach referred to as k-support anonymity to protect each sensitive item with k-1 other items of similar support. to achieve k-support anonymity, we introduce a pseudo taxonomy tree and have the third party mine the generalized frequent itemsets under the corresponding generalized association rules instead of association rules. the pseudo taxonomy is a construct to facilitate hiding of the original items, where each original item can map to either a leaf node or an internal node in the taxonomy tree. the rationale for this approach is that with a taxonomy tree, the k nodes to satisfy the k-support anonymity may be any k nodes in the taxonomy tree with the appropriate supports. so this approach can provide more candidates for k-support anonymity with limited fake items as only the leaf nodes, not the internal nodes, of the taxonomy tree need to appear in the transactions. otherwise for the association rule mining, the k nodes to satisfy the k-support anonymity have to correspond to the leaf nodes in the taxonomy tree. this is far more restricted. the challenge is thus on how to generate the pseudo taxonomy tree to facilitate k-support anonymity and to ensure the conservation of original frequent itemsets. the experimental results showed that our methods of k-support anonymity can achieve very good privacy protection with moderate storage overhead.
trust network inference for online rating data using generative models. in an online rating system, raters assign ratings to objects contributed by other users. in addition, raters can develop trust and distrust on object contributors depending on a few rating and trust related factors. previous study has shown that ratings and trust links can influence each other but there has been a lack of a formal model to relate these factors together. in this paper, we therefore propose trust antecedent factor (taf) model, a novel probabilistic model that generate ratings based on a number of rater's and contributor's factors. we demonstrate that parameters of the model can be learnt by collapsed gibbs sampling. we then apply the model to predict trust and distrust between raters and review contributors using a real data-set. our experiments have shown that the proposed model is capable of predicting both trust and distrust in a unified way. the model can also determine user factors which otherwise cannot be observed from the rating and trust data.
modeling relational events via latent classes. many social networks can be characterized by a sequence of dyadic interactions between individuals. techniques for analyzing such events are of increasing interest. in this paper, we describe a generative model for dyadic events, where each event arises from one of c latent classes, and the properties of the event (sender, recipient, and type) are chosen from distributions over these entities conditioned on the chosen class. we present two algorithms for inference in this model: an expectation-maximization algorithm as well as a markov chain monte carlo procedure based on collapsed gibbs sampling. to analyze the model's predictive accuracy, the algorithms are applied to multiple real-world data sets involving email communication, international political events, and animal behavior data.
topic models with power-law using pitman-yor process. one important approach for knowledge discovery and data mining is to estimate unobserved variables because latent variables can indicate hidden specific properties of observed data. the latent factor model assumes that each item in a record has a latent factor; the co-occurrence of items can then be modeled by latent factors. in document modeling, a record indicates a document represented as a "bag of words," meaning that the order of words is ignored, an item indicates a word and a latent factor indicates a topic. latent dirichlet allocation (lda) is a widely used bayesian topic model applying the dirichlet distribution over the latent topic distribution of a document having multiple topics. lda assumes that latent topics, i.e., discrete latent variables, are distributed according to a multinomial distribution whose parameters are generated from the dirichlet distribution. lda also models a word distribution by using a multinomial distribution whose parameters follows the dirichlet distribution. this dirichlet-multinomial setting, however, cannot capture the power-law phenomenon of a word distribution, which is known as zipf's law in linguistics. we therefore propose a novel topic model using the pitman-yor(py) process, called the py topic model. the py topic model captures two properties of a document; a power-law word distribution and the presence of multiple topics. in an experiment using real data, this model outperformed lda in document modeling in terms of perplexity.
data mining in the online services industry. the online services industry is a rapidly growing industry with a worldwide online ad market projected to grow from $48 billion in 2011 to $67 billion in 2013, of which 47% will come from display advertising and 53% from search advertising. online services division (osd) within microsoft is a leader in the consumer cloud space today with a strong portfolio of a set of 3 mutually reinforcing businesses: search, portal, advertising. they are supported by a shared foundational asset of intent & knowledge stores and a shared technology platform supporting large scale data and high performance systems. msn (portal) and bing (search) generate the content, traffic and data, that make for an exciting fertile environment for large scale data mining practice and system development. our advertisers are thus given more valuable targeting opportunities and better roi, which in turn, provide better economics, usability data, and allows for a higher quality services for our advertisers and experience for our users. the ability to transform data into meaningful, actionable insight is an important source of competitive advantage for osd. the data mining initiatives within the division continue to strive for excellence around the following goals: actionable insights through deep data analysis, data mining and data modeling at scale and with speed, increased productivity from deployed large scale data systems and tools, improved product and service development and decision making gained from effective measurement and experimentation, and a mature data culture in product teams that made the above possible. with many technical and data challenges ahead of us, we are committed to utilizing our huge data asset well to understand the need, intent, and behavior of our users for the purpose of serving them better.