OpenRaiser/Intern-Atlas huggingface dataset:
Statistics and Data Mining Techniques for Lifetime Value Modeling
Authors: D. R. Mani, James Drew, Andrew Betz, Piew Datta
Venue: KDD
Year: 1999

Customer lifetime value (LTV)-which is a measure of the profit generating potential, or value, of a customer-is increasingly being considered a touchstone for customer relationship management. The central challenge in predicting LTV is the production of estimated customer tenures with a given service supplier, based on information contained in company databases. Classical survival analysis techniques like proportional hazards regression focus on covariate effects frequently presumed to be linear, and examination of age-wise effects can be difficult. Further, segments of customers, whose lifetimes and covariate effects can vary widely, are not necessarily easy to detect. A new neural network model for hazard prediction is used to free proportional hazards-like models from their linearity and proportionality constraints, and clustering tools are applied to identify segments of customer hazard patterns. Using the proportional hazards and neural network models in tandem, we demonstrate how data mining tools can be apt complements of the classical statistical models, and show that their combined usage overcomes many of the shortcomings of each separate tool setresulting in a LTV tenure prediction model that is both accurate and understandable.
Prediction with Local Patterns using Cross-Entropy
Authors: Heikki Mannila, Dmitry Pavlov, Padhraic Smyth
Venue: KDD
Year: 1999

Sets of local patterns in the forms of rules and co-occurrence counts are produced by many data mining methods such as association rule algorithms.
Mining Lesion-Deficit Associations in a Brain Image Database
Authors: Vasileios Megalooikonomou, Christos Davatzikos, Edward Herskovits
Venue: KDD
Year: 1999

We present a data mining process for discovering associations between structures and functions of the human brain. Our approach is through the study of lesioned (abnormal) structures and associated functional deficits (disorders). For this purpose we have developed a BRAinImage Database (BRAID) that integrates image processing and visualization capabilities with statistical analysis of spatial and clinical data, providing access via extended SQL through a web interface.
Extending Naïve Bayes Classifiers Using Long Itemsets
Authors: Dimitris Meretakis, Beat Wüthrich
Venue: KDD
Year: 1999

Itemsets provide local descriptions of the data. This work proposes to use itemsets as basic means for classification purposes too. To enable this, the concept of class support SUpi of an itemset is introduced, i.e., how many times an itemset occurs when a specific class Ci is present. Class supports of frequent itemsets are computed in the training phase. Upon arrival of a new case to be classified, some of the generated itemsets are selected and their class supports SUpi are used to compute the probability that the case belongs to class cc The result is the class ci with highest such probability. We show that selecting and combining many and long itemsets providing new evidence (interesting) is an effective strategy for computing the class probabilities. The proposed classification technique is called Large Bayes as it happens to reduce to Naive Bayes classifier when all itemsets selected are of size one only. Experimental results on a large number of benchmark data sets show that Large Bayes consistently outperforms the widely used Ndive Bayes classifier. In many cases, Large Bayes is also superior to other state of the art classification methods such as C4.5, CBA (a recently proposed rule classifier built from association rules), and TAN (a Bayesian network extension of Ndive Bayes).
Using a Knowledge Cache for Interactive Discovery of Association Rules
Authors: Biswadeep Nag, Prasad Deshpande, David J. DeWitt
Venue: KDD
Year: 1999

Association rule mining is a valuable decision support technique that can be used to analyze customer preferences, buying patterns, and product correlations. Current systems are however handicapped by the long processing times required by mining algorithms that make them unsuitable for interactive use. In this paper, we propose the use of a knowledge cache that can reduce the response time by several orders of magnitude. Most of the performance gain comes from the idea of guaranteed support that allows us to completely eliminate database accesses in a large number of cases. Using this cache, the time taken to answer a query is proportional to just the size of the result, rather than to the size of the database. Cache replacement is best done by a benefit-metric based strategy that can easily adapt to changing query patterns.
Estimating Campaign Benefits and Modeling Lift
Authors: Gregory Piatetsky-Shapiro, Brij M. Masand
Venue: KDD
Year: 1999

In assessing the potential of data mining based marketing campaigns one needs to estimate the payoff of applying modeling to the problem of predicting behavior of some target population (e.g. attriters, people likely to buy product X, people likely to default on a loan, etc). This assessment has two components: a) the financial estimate of the campaign profitability, based on cost/benefit analysis and b) estimation of model accuracy in the targeted population using measures such as lift. We present a methodology for initial cost/benefit analysis and present surprising empirical results, based on actual business data from several domains, on achievable model accuracy. We conjecture that lift at T (where T is the target frequency) is usually about sqrt(1/T) for a good model. We also present formulae for estimating the entire lift curve and estimating expected profits.
2001: A Statistical Odyssey (Invited talk, Abstract only)
Authors: Daryl Pregibon
Venue: KDD
Year: 1999

This talk is an interim report on the 5 year plan launched in 1996 to provide a theoretical and computational foundation of Statistics for massive data sets. The plan coincided with the formation of AT&T Labs and the proposed research agenda of the InfoLab, which is both a physical laboratory and an interdisciplinary collection of information researchers in CS, mathematics, and Statistics. At the halfway point of this odyssey we can identify some success stories but more importantly it is an opportune time to re-calibrate the challenges and the milestones.
Efficient Progressive Sampling
Authors: Foster J. Provost, David D. Jensen, Tim Oates
Venue: KDD
Year: 1999

Having access to massive amounts of data does not necessarily imply that induction algorithms must use them all. Samples often provide the same accuracy with far less computational cost. However, the correct sample size rarely is obvious. We analyze methods for progressive samplingusing progressively larger samples as long as model accuracy improves. We explore several notions of efficient progressive sampling. We analyze efficiency relative to induction with all instances; we show that a simple, geometric sampling schedule is asymptotically optimal, and we describe how best to take into account prior expectations of accuracy convergence. We then describe the issues involved in instantiating an efficient progressive sampler, including how to detect convergence. Finally, we provide empirical results comparing a variety of progressive sampling methods. We conclude that progressive sampling can be remarkably efficient .
Scalable Algorithms for Mining Large Databases
Authors: Rajeev Rastogi, Kyuseok Shim
Venue: KDD
Year: 1999

A large number of corporations have invested heavily in information technology to manage their businesses more effectively, and vast amounts of critical business data have been stored in database systems. The volume of this data is expected to grow considerably in the near future. Yet many organizations have been unable to collect valuable insights from the data to guide their marketing strategy, investment and management policies. One of the reasons for this is that most information is stored implicitly in the large amounts of data. Fortunately, new and sophisticated techniques being developed in the area of data mining can help companies leverage their data more effectively and extract insightful information from their data. This tutorial describes the fundamental algorithms for data mining, many of which have been proposed in recent years. These techniques include association rules, correlation, causal relationships, clustering, outlier detection, similar time sequences, similar images, sequential patterns and classification.
Mining GPS Data to Augment Road Models
Authors: Seth Rogers, Pat Langley, Christopher Wilson
Venue: KDD
Year: 1999

Many advanced safety and navigation applications in vehicles require accurate, detailed digital maps, but manual lane measurements are expensive and time-consuming, making automated techniques desirable. This paper describes a data-mining approach to map refinement, using position traces that come from Global Positioning System receivers with differential corrections. The computed lane models enable safety applications, such as lanekeeping, and convenience applications, such as lane-changing advice. Experiments show that, starting from a baseline map that is commercially available, our lane models predict a vehicle's lane with high accuracy from a small number of passes over a particular road segment. Multiple position traces are a powerful new source of data that enables cheap, automated methods of inducing lane models, as well as other geographic knowledge, like traffic signals and elevations, and potentially impacts any geographic information system with a need to relate to actual behavior.
Interestingness via What is Not Interesting
Authors: Sigal Sahar
Venue: KDD
Year: 1999

As the size of mined databases increases, the sheer volume of rules produced by data-mining algorithms often overwhelms their users. These users only want to be presented with a short list of interesting rules. Some work has been conducted
Compressed Data Cubes for OLAP Aggregate Query Approximation on Continuous Dimensions
Authors: Jayavel Shanmugasundaram, Usama M. Fayyad, Paul S. Bradley
Venue: KDD
Year: 1999

of pre-computation are to be realized.
Monitoring a Newsfeed for Hot Topics
Authors: Mark Shewhart, Mark Wasson
Venue: KDD
Year: 1999

Article Monitoring a newsfeed for hot topics Share on Authors: Mark Shewhart LEXIS-NEXIS, a Division of Reed Elsevier plc, 9595 Springboro Pike, Miamisburg, Ohio LEXIS-NEXIS, a Division of Reed Elsevier plc, 9595 Springboro Pike, Miamisburg, OhioView Profile , Mark Wasson LEXIS-NEXIS, a Division of Reed Elsevier plc, 9595 Springboro Pike, Miamisburg, Ohio LEXIS-NEXIS, a Division of Reed Elsevier plc, 9595 Springboro Pike, Miamisburg, OhioView Profile Authors Info & Claims KDD '99: Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data miningAugust 1999 Pages 402–404https://doi.org/10.1145/312129.312301Online:01 August 1999Publication History 3citation548DownloadsMetricsTotal Citations3Total Downloads548Last 12 Months4Last 6 weeks0 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
A Study of Support Vectors on Model Independent Example Selection
Authors: Nadeem Ahmed Syed, Huan Liu, Kah Kay Sung
Venue: KDD
Year: 1999

As databases for real-world problems increase in size, there is a need in many situations to select and keep relevant training data for efficient storage and processing reasons. Support vector machines (SVMs) reportedly exhibit certain desirable properties in selecting and preserving useful training data as support vectors.
Handling Concept Drifts in Incremental Learning with Support Vector Machines
Authors: Nadeem Ahmed Syed, Huan Liu, Kah Kay Sung
Venue: KDD
Year: 1999

With the increase in the size of real-world databases, there is an ever-increasing need to scale up inductive learning algorithms.
Breaking the Barrier of Transactions: Mining Inter-Transaction Association Rules
Authors: Anthony K. H. Tung, Hongjun Lu, Jiawei Han, Ling Feng
Venue: KDD
Year: 1999

Most of the previous studies on mining association rules are on mining intro-transaction associations, i.e., the associations among items within the same transaction, where the notion of the transaction could be the items bought by the same customer, the events happened on the same day, etc. In this study, we break the barrier of transactions and extend the scope of mining association rules from traditional intratransaction associations to inter-transaction associations. Mining inter-transaction associations poses more challenges on efficient processing than mining intra-transaction associations because the number of potential association rules becomes extremely large after the boundary of transactions is broken. In this study, we introduce the notion of inter-transaction association rule, define its measurements: support and confidence, and develop an efficient algorithm, FITI (an acronym for "First Intra Then Inter"), for mining inter-transaction associations. We compare FITI with EH-Apriori, the best algorithm in our previous proposal, and demonstrate a substantial performance gain of FITI over EH- Apriori.
Evaluating a Class of Distance-Mapping Algorithms for Data Mining and Clustering
Authors: Jason Tsong-Li Wang, Xiong Wang, King-Ip (David) Lin, Dennis E. Shasha, Bruce A. Shapiro, Kaizhong Zhang
Venue: KDD
Year: 1999

A distance-mapping algorithm takes a set of objects and a distance metric and then maps those objects to a Euclidean or pseudo-Euclidean space in such a way that the distances among objects are approximately preserved. Distancemapping algorithms are a useful tool for clustering and visualization in data intensive applications, because they replace expensive distance calculations by sum-of-square calculations. This can make clustering in large databases with expensive distance metrics practical. In this paper we present five distance-mapping algorithms and conduct experiments to compare their performance in data clustering applications. These include two algorithms called FastMap and MetricMap, and three hybrid heuristics that combine the two algorithms in different ways. Experimental results on both synthetic and RNA data show the superiority of the hybrid algorithms. The results imply that FastMap and MetricMap capture complementary information about distance metrics and therefore ca...
Discovering Roll-Up Dependencies
Authors: Jef Wijsen, Raymond T. Ng, Toon Calders
Venue: KDD
Year: 1999

We introduce the problem of discovering functional determinacies that result from "rolling up" data to a higher abstraction level. Such a determinacy is called a Roll-Up Dependency (RUD). An example RUD is: The probability that two files in the same directory have the same file extension, is greater than a specific number. We show the applicability of RUDs for OLAP and data mining. We consider the problem of mining RUDs that satisfy specified support and confidence thresholds. This problem is NP-hard in the number of attributes.
Fast Density Estimation Using CF-Kernel for Very Large Databases
Authors: Tian Zhang, Raghu Ramakrishnan, Miron Livny
Venue: KDD
Year: 1999

This paper presents the CF-kernel method to estimate density functions from large datasets in a "best-efforts under the given resources" manner. By integrating the advantages of both the kernel method [WJ95] and the dynamical and incremental CF tree structure[ZRL96],
Depth first generation of long patterns
Authors: Ramesh C. Agarwal, Charu C. Aggarwal, V. V. V. Prasad
Venue: KDD
Year: 2000

In this paper we present an algorithm for mining long patterns in databases. The algorithm finds large itemsets by using depth first search on a lexicographic tree of itemsets. The focus of this paper is to develop CPU-efficient algorithms for finding frequent itemsets in the cases when the database contains patterns which are very wide. We refer to this algorithm as DepthProject, and it achieves more than one order of magnitude speedup over the recently proposed MaxMiner algorithm for finding long patterns. These techniques may be quite useful for applications in areas such as computational biology in which the number of records is relatively small, but the itemsets are very long. This necessitates the discovery of patterns using algorithms which are especially tailored to the nature of such domains.
The IGrid index: reversing the dimensionality curse for similarity indexing in high dimensional space
Authors: Charu C. Aggarwal, Philip S. Yu
Venue: KDD
Year: 2000

Article The IGrid index: reversing the dimensionality curse for similarity indexing in high dimensional space Share on Authors: Charu C. Aggarwal IBM T. J. Watson Research Center, Yorktown Heights, NY IBM T. J. Watson Research Center, Yorktown Heights, NYView Profile , Philip S. Yu IBM T. J. Watson Research Center, Yorktown Heights, NY IBM T. J. Watson Research Center, Yorktown Heights, NYView Profile Authors Info & Claims KDD '00: Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data miningAugust 2000 Pages 119–129https://doi.org/10.1145/347090.347116Online:01 August 2000Publication History 49citation752DownloadsMetricsTotal Citations49Total Downloads752Last 12 Months13Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Using the fractal dimension to cluster datasets
Authors: Daniel Barbará, Ping Chen
Venue: KDD
Year: 2000

Clustering is a widely used knowledge discovery technique. It helps uncovering structures in data that were not previously known. The clustering of large data sets has received a lot of attention in recent years, however, clustering is a still a challenging task since many published algorithms fail to do well in scaling with the size of the data set and the number of dimensions that describe the points, or in finding arbitrary shapes of clusters, or dealing effectively with the presence of noise. In this paper, we present a new clustering algorithm, based in the fractal properties of the data sets. The new algorithm, which we call Fractal Clustering (FC), places points incrementally in the cluster for which the change in the fractal dimension after adding the point is the least. This is a very natural way of clustering points, since points in the same cluster have a great degree of self-similarity among them (and much less self-similarity with respect to points in other clusters). FC requires one scan of the data, is suspendable at will, providing the best answer possible at that point, and is incremental. We show via experiments that FC effectively deals with large data sets, high-dimensionality and noise and is capable of recognizing clusters of arbitrary shape.
Multivariate discretization of continuous variables for set mining
Authors: Stephen D. Bay
Venue: KDD
Year: 2000

Many algorithms in data mining can be formulated as a set mining problem where the goal is to nd conjunctions or disjunctions of terms that meet user speci ed constraints. Set mining techniques have been largely designed for categorical or discrete data where variables can only take o n a xed numb e r o f v alues. However, many data sets also contain continuous variables and a common method of dealing with these is to discretize them by breaking them into ranges. Most discretization methods are univariate and consider only a single feature at a time sometimes in conjunction with the class variable. We argue that this is a sub-optimal approach for knowledge discovery as univariate discretization can destroy hidden patterns in data. Discretization should consider the e ects on all variables in the analysis and that two regions X and Y should only be in the same cell after discretization if the instances in those regions have similar multivariate distributions Fx Fy across all variables and combinations of variables. We present a b o ttom up merging algorithm to discretize continuous variables based on this rule. Our experiments indicate that the approach is feasible, that it does not destroy hidden patterns and that it generates meaningful intervals.
Automating exploratory data analysis for efficient data mining
Authors: Jonathan D. Becher, Pavel Berkhin, Edmund Freeman
Venue: KDD
Year: 2000

Having access to large data sets for the purpose of predictive data mining does not guarantee good models, even when the size of the training data is virtually unlimited. Instead, careful data preprocessing is required, including data cleansing, handling missing values, attribute representation and encoding, and generating derived attributes. In particular, the selection of the most appropriate subset of attributes to include is a critical step in building an accurate and efficient model. We describe an automated approach to the exploration, preprocessing, and selection of the optimal attribute subset whose goal is to simplify the KDD process and dramatically shorten the time to build a model. Our implementation finds inappropriate and suspicious attributes, performs target dependency analysis, determining optimal attribute encoding, generates new derived attributes, and provides a flexible approach to attribute selection. We present results generated by an industrial KDD environment called the Accrue Decision Series on several real world Web data sets.
Evolutionary algorithms in data mining: multi-objective performance modeling for direct marketing
Authors: Siddhartha Bhattacharyya
Venue: KDD
Year: 2000

Predictive models in direct marketing seek to identify individuals most likely to respond to promotional solicitations or other intervention programs. While standard modeling approaches embody single objectives, real-world decision problems often seek multiple performance measures. Decision-makers here desire solutions that simultaneously optimize on multiple objectives, or obtain an acceptable tradeoff amongst objectives.
A data mining framework for optimal product selection in retail supermarket data: the generalized PROFSET model
Authors: Tom Brijs, Bart Goethals, Gilbert Swinnen, Koen Vanhoof, Geert Wets
Venue: KDD
Year: 2000

In recent years, data mining researchers have developed efficient association rule algorithms for retail market basket analysis. Still, retailers often complain about how to adopt association rules to optimize concrete retail marketing-mix decisions. It is in this context that, in a previous paper, the authors have introduced a product selection model called PROFSET. This model selects the most interesting products from a product assortment based on their cross-selling potential given some retailer defined constraints. However this model suffered from an important deficiency: it could not deal effectively with supermarket data, and no provisions were taken to include retail category management principles. Therefore, in this paper, the authors present an important generalization of the existing model in order to make it suitable for supermarket data as well, and to enable retailers to add category restrictions to the model. Experiments on real world data obtained from a Belgian supermarket chain produce very promising results and demonstrate the effectiveness of the generalized PROFSET model.