waashk/dblp huggingface dataset:
maximizing classifier utility when training data is costly. classification is a well-studied problem in machine learning and data mining. classifier performance was originally gauged almost exclusively using predictive accuracy. however, as work in the field progressed, more sophisticated measures of classifier utility that better represented the value of the induced knowledge were introduced. nonetheless, most work still ignored the cost of acquiring training examples, even though this affects the overall utility of a classifier. in this paper we consider the costs of acquiring the training examples in the data mining process; we analyze the impact of the cost of training data on learning, identify the optimal training set size for a given data set, and analyze the performance of several progressive sampling schemes, which, given the cost of the training data, will generate classifiers that come close to maximizing the overall utility.
biokdd06: data mining in bioinformatics. data mining is the process of automatic discovery of novel and understandable models and patterns from large amounts of data. bioinformatics is the science of storing, analyzing, and utilizing information from biological data such as sequences, molecules, gene expressions, and pathways. development of novel data mining methods will play a fundamental role in understanding these rapidly expanding sources of biological data. the extensive databases of biological information create both challenges and opportunities for developing novel data mining methods.
relevance search and anomaly detection in bipartite graphs. many real applications can be modeled using bipartite graphs, such as users vs. files in a p2p system, traders vs. stocks in a financial trading system, conferences vs. authors in a scientific publication network, and so on. we introduce two operations on bipartite graphs: 1) identifying similar nodes (relevance search), and 2) finding nodes connecting irrelevant nodes (anomaly detection). and we propose algorithms to compute the relevance score for each node using random walk with restarts and graph partitioning; we also propose algorithms to identify anomalies, using relevance scores. we evaluate the quality of relevance search based on semantics of the datasets, and we also measure the performance of the anomaly detection algorithm with manually injected anomalies. both effectiveness and efficiency of the methods are confirmed by experiments on several real datasets.
on the medical frontier: the 2006 kdd cup competition and results. the 2006 kdd cup competition featured three data mining tasks drawn from a medical imaging domain. at the core, all of these tasks were concerned with identifying pulmonary embolisms (pes) from pre-processed computed tomography (ct) images of human lungs. however, these tasks were complicated by features such as multi-instance learning, stringent performance standards, hard-threshold evaluation functions, spatial correlations, and small training sets. this paper gives an overview of the data and medical imaging tasks, the competition and evaluation, and the competition victors.
privacy-enhanced linking. while computer scientists are uniquely situated to incorporate privacy protections in the link analysis algorithms they construct, most computer scientists are unaware of this opportunity and of ways to think about achieving needed protections. the work presented in this writing introduces a new way for computer scientists to think about providing privacy protection within link analysis and introduces the notion of "privacy-enhanced linking" as algorithms that perform link analysis with guarantees of privacy protection modeled after the fair information practices. in this approach, privacy protection is realized by assessing the validity and interpretation of link analysis results such that inappropriate harm to individuals is provably minimized.
data mining for business applications: kdd-2006 workshop. even though data mining has been successful in becoming a major component of various business processes as well as in transferring innovations from academic research into the business world, the gap between the problems that the research community works on and real-world ones is still significant. we believe that it is essential for the business and the academic research communities to interact frequently. the goal of the kdd-2006 workshop on data mining for business applications was to gather both researchers and business practitioners and talk about their different perspectives and to share their latest problems and idea. we wanted to not only bring them together at kdd but also to create relationships that would continue and grow after the event as well.
data mining, national security, privacy and civil liberties. in this paper, we describe the threats to privacy that can occur through data mining and then view the privacy problem as a variation of the inference problem in databases.
ubdm 2006: utility-based data mining 2006 workshop report. in this report we provide a summary of the second international workshop on utility-based data mining (ubdm-06) held in conjunction with the 12th acm sigkdd international conference on knowledge discovery and data mining. the workshop was held on august 20, 2006 in philadelphia, pa, usa. as was the case for the first ubdm workshop, this workshop brought together researchers who currently investigate how to incorporate economic utility aspects into the data mining process and practitioners who have real-world experience with how these factors influence data mining applications.
frequent closed itemset based algorithms: a thorough structural and analytical survey. as a side effect of the digitalization of unprecedented amount of data, traditional retrieval tools proved to be unable to extract hidden and valuable knowledge. data mining, with a clear promise to provide adequate tools and/or techniques to do so, is the discovery of hidden information that can be retrieved from datasets. in this paper, we present a structural and analytical survey of <u>f</u>requent <u>c</u>losed <u>i</u>temset (fci) based algorithms for mining association rules. indeed, we provide a structural classification, in four categories, and a comparison of these algorithms based on criteria that we introduce. we also present an analytical comparison of fci-based algorithms using benchmark dense and sparse datasets as well as "worst case" datasets. aiming to stand beyond classical performance analysis, we intend to provide a focal point on performance analysis based on memory consumption and advantages and/or limitations of optimization strategies, used in the fci-based algorithms.
the third sigkdd workshop on mining temporal and sequential data (kdd/tdm 2004). in this short report, we provide a summary of the results, issues, and research directions on mining temporal and sequential data, discussed in tdm-2004, held in conjunction with the 10-th acm sigkdd international conference on knowledge discovery and data mining (kdd-2004) on august 22, 2004 in seattle, washington, u.s.a.
parameter optimized, vertical, nearest-neighbor-vote and boundary-based classification. in this paper, we describe a reliable high performance classification system that includes a nearest neighbor vote based classification and a local decision boundary based classification combined with an evolutionary algorithm for parameter optimization and a vertical data structure (predicate tree or p-tree1 for processing efficiency.
predicting the effects of gene deletion. in this paper, we describe techniques that can be used to predict the effects of gene deletion. we will focus mainly on the creation of predictive variables, and then briefly discuss different modeling techniques that have been used successfully on this data.
webkdd 2006: web mining and web usage analysis post-workshop report. in this report, we summarize the contents and outcomes of the recent webkdd 2006 workshop on web mining and web usage analysis that was held in conjunction with the 12th acm sigkdd international conference on knowledge discovery and data mining (kdd 2006), august 20-23, 2006, in philadelphia, pennsylvania. in 2006, webkdd was organized for the eighth time. it solicited papers on the broad overview subject "knowledge discovery on the web" and on new directions for web mining research. we reflect on the results of the workshop, as captured in presentations and plenary discussions.
unsupervised pattern mining from symbolic temporal data. we present a unifying view of temporal concepts and data models in order to categorize existing approaches for unsupervised pattern mining from symbolic temporal data. in particular we distinguish time point-based methods and interval-based methods as well as univariate and multivariate methods. the mining paradigms and the robustness of many proposed approaches are compared to aid the selection of the appropriate method for a given problem. for time points, sequential pattern mining algorithms can be used to express equality and order of time points with gaps in multivariate data. for univariate data and limited gaps suffix tree methods are more efficient. recently, efficient algorithms have been proposed to mine the more general concept of partial order from time points. for time interval data with precise start and end points the relations of allen can be used to formulate patterns. the recently proposed time series knowledge representation is more robust on noisy data and offers an alternative semantic that avoids ambiguity and is more expressive. for both pattern languages efficient mining algorithms have been proposed.
interview with simon funk. interview with simon funk -- a netflix prize leader, an outstanding hacker, and an original thinker.
anti-matter detection: particle physics model for kdd cup 2004. what is the difference between matter and anti-matter? a. i. insight's winning solution on the particle physics task for the 2004 kdd cup demonstrates how an accurate predictive model can be formulated without knowledge of the content of the data. information on the data was not available for the modeling, including a description on the outcome to be predicted. in other words, an 80 x 150,000 grid of numbers with the header "particle physics" was all that was given to the 500+ registrants of this competition. key steps in creating the winning model were interactive analysis of the variables, detection of interactions, a powerful self-organizing neural network, and customization of the 4 different error criteria.
the problem of disguised missing data. missing data is a well-recognized problem in large datasets, widely discussed in the statistics and data analysis literature. many programming environments provide explicit codes for missing data, but these are not standardized and are not always used. this lack of standardization is one of the leading causes of the subtle problem of disguised missing data, in which unknown, inapplicable, or otherwise nonspecified responses are encoded as valid data values. following a brief overview of the problem of explicitly coded missing data, this paper discusses sources, consequences, and detection of disguised missing data, including two real-world examples. as the first of these examples illustrates, the consequences of disguised missing data can be quite serious. the key to its detection lies in first, recognizing disguised missing data as a possibility and second, finding a sufficiently informative view of the data to reveal its presence.
protein matching with custom neural network objective functions. this 2004 kdd cup presents a perfect case where the usual neural network objective functions do not apply. while the contest problem consisted of 4 different entries with 4 different objective functions, this paper will focus on the solution optimizing grmse (grouped root mean squared error). it will be shown that the more typical objective functions (including rmse) cannot be as effective at meeting this criteria. while this objective function may be specific to this problem, and the reader may never see this exact function again in his/her lifetime, the idea behind this paper is applicable in many situations. too often neural networks are used to minimize sse (sum of the squares of the errors) or cross entropy, when the true measure of success for the model may require a small coding change to the neural network objective function. it is shown in this paper that a few small coding changes can make a big difference on a model's performance.
kdd/mdm 2006: the 7 kdd multimedia data mining workshop report. this is a report of the 7th acm kdd/mdm workshop on multimedia data mining held in philadelphia, pa, usa, on 20 august 2006. this report outlines the motivations to hold the workshop, the response we have received in terms of paper submissions and participations, as well as the post-workshop events. the central theme of this year's mdm workshop is to merge multimedia and data mining research together to exploit the synergy between the two areas to advance and promote the research and development in multimedia data mining. the workshop was extremely successful. the feedback we have received during and after the workshop was overwhelmingly positive, which indicates the emerging interest and enthusiasm on the topic of multimedia data mining in the related research communities. due to this fact, we plan to continue this series next year.
can exclusive clustering on streaming data be achieved? clustering on streaming data aims at partitioning a list of data points into k groups of "similar" objects by scanning the data once. most current one-scan clustering algorithms do not keep original data in the resulting clusters. the output of the algorithms is therefore not the clustered data points but the approximations of data properties according to the predefined similarity function, such that k centers and radiuses reflect the up-to-date data grouping. in this paper, we raise a critical question: can the partition-based clustering, or exclusive clustering, be achieved on streaming data by those currently available algorithms? after identifying the differences between traditional clustering and clustering on data streams, we discuss the basic requirements for the clusters that can be discovered from streaming data. we evaluate the recent work that is based on a subcluster maintenance approach. by using a few straightforward examples we illustrate that the subcluster maintenance approach may fail to resolve the exclusive clustering on data streams. based on our observations, we also present the challenges on any heuristic method that claims solving the clustering problem on data streams in general.
kdd physics task: discussion of modeling approaches. in this paper, we present the methodology followed by inductis in developing the predictive models for quantum physics task in kdd cup 2004. we discuss many challenges that we faced in approaching the task and how we overcame them. we explored the solution space with various classification approaches and finally used stochastic gradient boosting offered on the treenet platform. a set of treenet models was fit varying various parameters and its performance was measured.
revenue recovering with insolvency prevention on a brazilian telecom operator. this paper deals with a real application on a brazilian telephone company. data mining analysis, based on neural networks, was performed on the customer base in order to understand and to prevent bad debt events. this paper describes the knowledge discovering process and focuses on two main products: the cluster analysis of the customer base and a bad debt event classification model. the segmentation of the customer base has provided a better understanding of several groups of customers' behavior. distinct actions are taken, depending on the segment a given client was put in, according to strategic directions of the company. the classification of insolvent customers is used as a tool to help the company to take preventing actions in order to avoid main losses and taxes leakage. the results of the project's implantation show that investment on information technology infra-structure for data mining is highly profitable.
state of the art of graph-based data mining. the need for mining structured data has increased in the past few years. one of the best studied data structures in computer science and discrete mathematics are graphs. it can therefore be no surprise that graph based data mining has become quite popular in the last few years.this article introduces the theoretical basis of graph based data mining and surveys the state of the art of graph-based data mining. brief descriptions of some representative approaches are provided as well.
classifying search engine queries using the web as background knowledge. the performance of search engines crucially depends on their ability to capture the meaning of a query most likely intended by the user. we study the problem of mapping a search engine query to those nodes of a given subject taxonomy that characterize its most likely meanings. we describe the architecture of a classification system that uses a web directory to identify the subject context that the query terms are frequently used in. based on its performance on the classification of 800,000 example queries recorded from msn search, the system received the runner-up award for query categorization performance of the kdd cup 2005.
mining with rarity: a unifying framework. rare objects are often of great interest and great value. until recently, however, rarity has not received much attention in the context of data mining. now, as increasingly complex real-world problems are addressed, rarity, and the related problem of imbalanced data, are taking center stage. this article discusses the role that rare classes and rare cases play in data mining. the problems that can result from these two forms of rarity are described in detail, as are methods for addressing these problems. these descriptions utilize examples from existing research. so that this article provides a good survey of the literature on rarity in data mining. this article also demonstrates that rare classes and rare cases are very similar phenomena---both forms of rarity are shown to cause similar problems during data mining and benefit from the same remediation methods.
kdd workshop on data mining standards, services & platforms (dm-ssp) 2006. dm-ssp '06 was the fourth year there has been a conference on data mining standards, services and platforms and the sixth year that there has been a conference on the predictive model markup language (pmml) and related areas. the workshop consisted of five talks and two panels.
report on ubdm-05: workshop on utility-based data mining. in this report we provide a summary of the first international workshop on utility-based data mining (ubdm-05) held in conjunction with the 11th acm sigkdd international conference on knowledge discovery and data mining. the workshop was held on august 21, 2005 in chicago, il, usa. this workshop was geared toward researchers with an interest in how economic utility factors affect data mining (e.g., researchers in cost-sensitive learning and active learning) and practitioners who have real-world experience with how these factors influence data mining applications.
automatic microscopy video mining for leukocytes. biological videos are very different from conventional videos. automatic spatiotemporal mining of moving cells from in vivo microscopy videos is extremely difficult because of the severe noises, camera/subject movements, deformations, and strong dependencies on microscopy operators. in this paper, we present an automatic spatiotemporal mining system of rolling and adherent leukocytes for intravital videos. the magnitude of leukocyte adhesion and decrease in rolling velocity are common interests in inflammation response studies. currently, there is no existing system which is perfect for such purposes. several approaches have been proposed for tracking leukocytes. however, these approaches can either only track leukocytes that roll along the centerline of the blood vessel, or can only handle leukocytes with fixed morphologies. in addition, the camera/subject movement is a severe problem which occurs frequently while analyzing in vivo microscopy videos. in this paper, we proposed a new method for automatic recognition of non-adherent and adherent leukocytes. the proposed method includes three steps: (1) camera/subject movement alignment; (2) moving leukocytes detection; (3) adherent leukocytes detection. the experimental results demonstrate the effectiveness of the proposed method.
graphical modeling based gene interaction analysis for microarray data. dna microarray provides a powerful basis for analysis of gene expression. data mining methods such as clustering have been widely applied to microarray data to link genes that show similar expression patterns. however, this approach usually fails to unveil gene-gene interactions in the same cluster. in this paper, we propose to use graphical modeling based interaction analysis for this purpose. we apply graphical gaussian model to discover pairwise gene interactions and use loglinear model to discover multi-gene interactions. we have constructed a prototype system that permits rapid interactive exploration of gene relationships; results can be validated by experts or known information, or suggest new experiments. we have tested our methodology using the yeast microarray data. our results reveal some previously unknown interactions that have solid biological explanations.
citation prediction using time series approach kdd cup 2003 (task 1). in this article we describe our experiences in building the winning system for kdd cup, 2003, task 1. this year's competition was based on a very large archive of research papers that provides an unusually comprehensive snapshot of a particular social network in action; in addition to the full text of research papers, it includes both explicit citation structure and partial data on the downloading of papers by users. it provides a framework for testing general network and usage mining techniques, which can be explored via four varied and interesting tasks. each task is a separate competition with its own specific goal. in task 1 the goal is to predict the change in number of citations to each paper in the archive over time.the contest was very challenging because the given data was not in a format suitable for conventional data mining techniques. so we had to do a considerable amount of data processing. also there were different sources of data like tex files, citation graph, slac-data database. so we had to make a decision about which sources to use and how much to use.
background and overview for kdd cup 2002 task 1: information extraction from biomedical articles. this paper presents a background and overview for task 1 (of 2 tasks) of the kdd challenge cup 2002, a competition held in conjunction with the acm sigkdd international conference on knowledge discovery and data mining (kdd), july 23--26, 2002. task 1 dealt with detecting which papers, in a set of fruitfly genetics papers (texts), contained experimental results about gene products (transcripts and proteins), and also within each paper, which genes had experimental results about their products mentioned.
drug exposure side effects from mining pregnancy data. this paper presents an interdisciplinary collaborative research project between the epidemiology department and the computer science department for using data mining technique to analyze data from pregnant women. specifically, the authors use association rule mining approach to derive possible side effects due to exposure to multiple drugs at different duration of the pregnancy. the derived temporal sequential rules discover new information that would not be detected by the traditional analysis method that is currently used in pharmaco-epidemiology.