waashk/dblp huggingface dataset:
mdm/kdd: multimedia data mining for the second time. this is brief report summarizes the presentations, conclusions and directions for future work that were discussed during the second edition of the international workshop on multimedia data mining. the report includes references to resources where one can find more information about the workshop format, the proceedings and the workshop participants.
enabling more sophisticated gene expression analysis for understanding diseases and optimizing treatments. we survey the progress in the analysis of gene expression data for the purposes of disease subtype diagnosis, new subtype discovery, and understanding of diseases and treatment responses. we find existing works fall short on several issues: these works provide little information on the interplay between selected genes; the collection of pathways that can be used, evaluated, and ranked against the observed expression data is limited; and a comprehensive set of rules for reasoning about relevant molecular events has not been compiled and formalized. we thus envision an advanced integrated framework, and are developing a system based on it, to provide biologically inspired solutions. it comprises: (i) automated analysis and extraction of information from biomedical texts; (ii) targeted construction of known pathways; and (iii) direct hypothesis generation based on logical reasoning on, and tests for, consistencies and inconsistencies of observed data against known pathways.
novel approaches for small biomolecule classification and structural similarity search. structural similarity search among small molecules is a standard tool used in molecular classification and in-silico drug discovery. the effectiveness of this general approach depends on how well the following problems are addressed. the notion of similarity should be chosen for providing the highest level of discrimination of compounds with respect to the bioactivity of interest. the data structure for performing search should be very efficient as the molecular databases of interest include several millions of compounds. in this paper we summarize the recent applications of k-nearest-neighbor search method for small molecule classification. the k-nn classification of small molecules is based on selecting the most relevant set of chemical descriptors which are then compared under standard minkowski distance lp. here we describe how to computationally design the optimal weighted minkowski distance wlp for maximizing the discrimination between active and inactive compounds wrt bioactivities of interest. k-nn classification requires fast similarity search for predicting bioactivity of a new molecule. we then focus on construction of pruning based k-nn search data structures for any wlp distance that minimizes similarity search time. the accuracy achieved by k-nn classifier is better than the alternative lda and mlr approaches and is comparable to the ann methods. in terms of running time, k-nn classifier is considerably faster than the ann approach especially when large data sets are used. furthermore, k-nn classifier is capable of quantification of the level of bioactivity rather than returning a binary decision and can bring more insight to the nature of the activity via eliminating unrelated descriptors of the compounds with respect to the activity in question.
data mining and audience intelligence for advertising. growth in the global advertising industry - especially the recent rapid growth in online advertising - has generated large volumes of data, bringing along with it many challenging data mining problems. researchers from various disciplines have brought their expertise to solve these exciting problems, leading to a plethora of novel applications and new algorithms. we strongly felt that we needed a forum where data mining researchers and practitioners, from both academia and the industry, could come together to share their experience on advertising. to this end, we organized adkdd 2007 1, the first international workshop on data mining and audience intelligence for advertising, in conjunction with kdd 2007 at san jose, california, usa. in this report, we will present a summary of the workshop.
report on biokdd04: workshop on data mining in bioinformatics. biokdd'04 was held in conjunction with the 10th acm sigkdd international conference on knowledge discovery and data mining, in seattle, wa, in august 2004. there are numerous sources of biological data that provides challenging opportunities for data mining. for example, the structural genomics initiative aims to catalog the structure-function information for proteins. advances in technology such as microarrays have launched the subfield of genomics and proteomics to study the genes, proteins, and the regulatory gene expression circuitry inside the cell. other sources of data include the rapidly growing literature in bioinformatics (e.g., pubmed), the data on biochemical pathways, the evolutionary relationships among organisms in the tree of life, high throughput drug design combinatorial libraries, and so on. in addition to the data from biology and genomics, there are rich sources of data from other biosciences, including biomedical, and neuroscience data.
visa: visual subspace clustering analysis. to gain insight into today's large data resources, data mining extracts interesting patterns. to generate knowledge from patterns and benefit from human cognitive abilities, meaningful visualization of patterns are crucial. clustering is a data mining technique that aims at grouping data to patterns based on mutual (dis)similarity. for high dimensional data, subspace clustering searches patterns in any subspace of the attributes as patterns are typically obscured by many irrelevant attributes in the full space. for visual analysis of subspace clusters, their comparability has to be ensured. existing subspace clustering approaches, however, lack interactive visualization and show bias with respect to the dimensionality of subspaces. in this work, dimensionality unbiased subspace clustering and a novel distance function for subspace clusters are proposed. we suggest two visualization techniques that allow users to browse the entire subspace clustering, to zoom into individual objects, and to analyze subspace cluster characteristics in-depth. bracketing of different parameter settings enable users to immediately see the effect of parameters on their data and hence to choose the best clustering result for further analysis. usage of user analysis for feedback to the subspace clustering algorithm directly improves the subspace clustering. we demonstrate our visualization techniques on real world data and confirm results through additional accuracy measurements and comparison with existing subspace clustering algorithms.
biokdd01: workshop on data mining in bioinformatics. in this report we provide a summary of the biokdd01 workshop on data mining in bioinformatics, held in conjunction with the 7th acm sigkdd international conference on knowledge discovery and data mining, august 26, 2001 at san francisco, california, usa.
more bang for their bucks: assessing new features for online advertisers. online search systems that display ads continually offer new features that advertisers can use to fine-tune and enhance their ad campaigns. an important question is whether a new feature actually helps advertisers. in an ideal world for statisticians, we would answer this question by running a statistically designed experiment. but that would require randomly choosing a set of advertisers and forcing them to use the feature, which is not realistic. accordingly, in the real world, new features for advertisers are seldom evaluated with a traditional experimental protocol. instead, customer service representatives select advertisers who are invited to be among the first to test a new feature (i.e., white-listed), and then each white-listed advertiser chooses whether or not to use the new feature. neither the customer service representative nor the advertiser chooses at random. this paper addresses the problem of drawing valid inferences from whitelist trials about the effects of new features on advertiser happiness. we are guided by three principles. first, statistical procedures for whitelist trials are likely to be applied in an automated way, so they should be robust to violations of modeling assumptions. second, standard analysis tools should be preferred over custom-built ones, both for clarity and for robustness. standard tools have withstood the test of time and have been thoroughly debugged. finally, it should be easy to compute reliable confidence intervals for the estimator. we review an estimator that has all these attributes, allowing us to make valid inferences about the effects of a new feature on advertiser happiness.
kdd cup and workshop 2007. the kdd cup is the oldest of the many data mining competitions that are now popular [1]. it is an integral part of the annual acm sigkdd international conference on knowledge discovery and data mining (kdd). in 2007, the traditional kdd cup competition was augmented with a workshop with a focus on the concurrently active netflix prize competition [2]. the kdd cup itself in 2007 consisted of a prediction competition using netflix movie rating data, with tasks that were different and separate from those being used in the netflix prize itself. at the workshop, participants in both the kdd cup and the netflix prize competition presented their results and analyses, and exchanged ideas.
trajectory-based visual analysis of large financial time series data. visual analytics seeks to combine automatic data analysis with visualization and human-computer interaction facilities to solve analysis problems in applications characterized by occurrence of large amounts of complex data. the financial data analysis domain is a promising field for research and application of visual analytics technology, as it prototypically involves the analysis of large data volumes in solving complex analysis tasks. we introduce a visual analytics system for supporting the analysis of large amounts of financial time-varying indicator data. a system, driven by the idea of extending standard technical chart analysis from one to two-dimensional indicator space, is developed. the system relies on an unsupervised clustering algorithm combined with an appropriately designed movement data visualization technique. several analytical views on the full market and specific assets are offered for the user to navigate, to explore, and to analyze. the system includes automatic screening of the potentially large visualization space, preselecting possibly interesting candidate data views for presentation to the user. the system is applied to a large data set of time varying 2-d stock market data, demonstrating its effectiveness for visual analysis of financial data. we expect the proposed techniques to be beneficial in other application areas as well.
correlating summarization of multi-source news with k-way graph bi-clustering. with the emergence of enormous amount of online news, it is desirable to construct text mining methods that can extract, compare and highlight similarities of them. in this paper, we explore the research issue and methodology of correlated summarization for a pair of news articles. the algorithm aligns the (sub)topics of the two news articles and summarizes their correlation by sentence extraction. a pair of news articles are modelled with a weighted bipartite graph. a mutual reinforcement principle is applied to identify a dense subgraph of the weighted bipartite graph. sentences corresponding to the subgraph are correlated well in textual content and convey the dominant shared topic of the pair of news articles. as a further enhancement for lengthy articles, a k-way bi-clustering algorithm can first be used to partition the bipartite graph into several clusters, each containing sentences from the two news reports. these clusters correspond to shared subtopics, and the above mutual reinforcement principle can then be applied to extract topic sentences within each subtopic group.
mining motion patterns using color motion map clustering. automatically extracting previously unknown behavior patterns from videos that track animals with various physical conditions can accelerate our understanding of animal behaviors and their influential factors, resulting in major medical and economic benefits. unfortunately, extracting behavior patterns from videos recordings remains as a very challenging task due to their extensive duration and the unstructured natures. this task is further complicated in a completely darken animal cage with inconsistent infrared lighting, moving reflections, or other cage debris such as the cage bedding. in this research, we propose a new motion model that enables us to measure the similarities among different animal movements in high precision so a clustering method can correctly separate recurring movements from infrequent random movements. more specifically, our model first transforms the spatial and temporal features of animal movements into a sequence of color images, referred to as color motion maps (cmms). the task of mining recurring behavior patterns is then reduced to clustering similar color images in a database. we will use a real infrared video to demonstrate the capability of our model in capturing distinguished but brief animal movements that are embedded within a sequence of other animal movements.
market basket recommendations for the hp smb store. the analytics team at hewlett-packard recently executed a manually-driven cross-sell/up-sell pilot in the small and medium business online store and call center. the pilot, for which management dictated a 1 month development timeframe, utilized sales transaction, product configuration, and product availability data. leveraging market basket analysis techniques among a small subset of available product skus, the pilot yielded a roi of more than $300k/month and more importantly, gave birth to greater opportunities to further showcase the power of analytics and data driven decision-making at hp.
visual analysis of dynamic group membership in temporal social networks. c-group is a tool for analyzing dynamic group membership in temporal social networks over time. unlike most network visualization tools, which show the group structure within an entire network, or the group membership for a single actor, c-group allows users to focus their analysis on a pair of individuals. while c-group allows for viewing the addition and deletion of nodes (actors) and edges (relationships) over time, its major contribution is its focus on changing group memberships over time. by doing so, users can investigate the context of temporal group memberships for the pair. c-group provides users with a flexible interface for defining (and redefining) groups interactively, and supports two novel visual representations of the evolving group memberships. this flexibility gives users alternate views that are appropriate for different network sizes and provides users with different insights into the grouping behavior. we demonstrate the utility of the tool on a scientific publication network.
interview with jon kleinberg. interview with jon kleinberg, a pioneer in web mining, social network analysis, and other fields and a winner of many awards, including 2 kdd best papers and a macarthur 'genius' award.
predicting who rated what in large-scale datasets. kdd cup 2007 focuses on movie rating behaviors. the goal of the task "who rated what" is to predict whether "existing" users will review "existing" movies in the future. we cast the task as a link prediction problem and address it via a simple classification approach. compared with other applications for link prediction, there are two major challenges in our task: (1) the huge size of the netflix data; (2) the prediction target is complicated by many factors, such as a general decrease of interest in old movies and more tendency to review more movies by netflix users due to the success of the internet dvd rental industries. we address the first challenge by "selective" subsampling and the second by combining information from the review scores, movie contents and graph topology effectively.
pinkdd'07: privacy, security, and trust in kdd post-workshop report. in this report, we summarize the events of the first international workshop on privacy, security, and trust in kdd (pinkdd), which was held in conjunction with the 13th acm sigkdd international conference on knowledge discovery and data mining. the workshop convened on august 12, 2007 in san jose, california and brought together researchers, as well as practitioners, working on how privacy, security, and trust can be resolved or modeled within a data mining framework.
dddm2007: domain driven data mining. real-world data mining generally must consider and involve domain and business oriented factors such as human knowledge, constraints and business expectations. this encourages the development of a domain driven methodology to strengthen data-centered pattern mining. this report presents a review of the acm sigkdd workshop on domain driven data mining (dddm2007), held in conjunction with the thirteenth acm sigkdd international conference on knowledge discovery and data mining (kdd07), which was held in san jose, usa on 12 august, 2007. the aims and objectives of this workshop were to provide a premier forum for sharing innovative findings, knowledge, insights, experiences and lessons in tackling challenges met in domain driven, actionable knowledge discovery in the real world.
managing discoveries in the visual analytics process. visualization systems traditionally focus on graphical representation of information. they tend not to provide integrated analytical services that could aid users in tackling complex knowledge discovery tasks. users' exploration in such environments is usually impeded due to several problems: 1) valuable information is hard to discover when too much data is visualized on the screen; 2) users have to manage and organize their discoveries off line, because no systematic discovery management mechanism exists; 3) their discoveries based on visual exploration alone may lack accuracy; and 4)they have no convenient access to the important knowledge learned by other users. to tackle these problems, it has been recognized that analytical tools must be introduced into visualization systems. in this paper, we present a novel analysis-guided exploration system, called the nugget management system (nms). it leverages the collaborative effort of human comprehensibility and machine computations to facilitate users' visual exploration processes. specifically, nms first helps users extract the valuable information (nuggets) hidden in datasets based on their interests. given that similar nuggets may be rediscovered by different users, nms consolidates the nugget candidate set by clustering based on their semantic similarity. to solve the problem of inaccurate discoveries, localized data mining techniques are applied to refine the nuggets to best represent the captured patterns in datasets. visualization techniques are then employed to present our collected nugget pool and thus create the nugget view. based on the nugget view, interaction techniques are designed to help users observe and organize the nuggets in a more intuitive manner and eventually faciliate their sense-making process. we integrated nms into xmdvtool, a freeware multivariate visualization system. user studies were performed to compare the users' efficiency and accuracy in finishing tasks on real datasets, with and without the help of nms. our user studies confirmed the effectiveness of nms.
major components of the gravity recommendation system. the netflix prize is a collaborative filtering problem. this subfield of machine learning became popular in the late 1990s with the spread of online services that used recommendation systems (e.g. amazon, yahoo! music, and of course netflix). the aim of such a system is to predict what items a user might like based on his/her and other users' previous ratings. the netflix prize dataset is much larger than former benchmark datasets, therefore the scalability of the algorithms is a must. this paper describes the major components of our blending based solution, called the gravity recommendation system (grs). in the netflix prize contest, it attained rmse 0.8743 as of november 2007. we now compare the effectiveness of some selected individual and combined approaches on a particular subset of the prize dataset, and discuss their important features and drawbacks.
kdd cup 2007 task 1 winner report. kdd cup 2007 focuses on predicting aspects of movie rating behavior. we present our prediction method for task 1 "who rated what in 2006" where the goal is to predict which users rated which movies in 2006. we use the combination of the following methods, listed in the order of their accuracy: &bull; the predicted number of ratings for each movie based on time series analysis, also using movie and dvd release dates and movie series detection by the edit distance of the titles. &bull; the predicted number of ratings by each user by using the fact that ratings were sampled proportional to the margin. &bull; the low rank approximation of the 0-1 matrix of known user-movie pairs with rating. &bull; prediction by using the movie-movie similarity matrix. &bull; association rules obtained by frequent sequence mining of user ratings considered as ordered itemsets. by combining the predictions by linear regression we obtained a prediction with root mean squared error 0.256. the first runner up result was 0.263 while a pure all zeroes prediction already gives 0.279, indicating the hardness of the task.
a combination of approaches to solve task "how many ratings" of the kdd cup 2007. this paper presents a solution to the kdd cup 2007 task "how many ratings?". the combination of three different approaches is used to produce a final solution which improves the results obtained by each of these procedures by itself.
lessons from the netflix prize challenge. this article outlines the overall strategy and summarizes a few key innovations of the team that won the first netflix progress prize.
introduction. the information overload is a well-known phenomenon of the information age, since due to the progress in computer power and storage capacity over the last decades, data is produced at an incredible rate, and our ability to collect and store these data is increasing at a faster rate than our ability to analyze it. but, the analysis of these massive, typically messy and inconsistent, volumes of data is crucial in many application domains. for decision makers, analysts or emergency response teams it is an essential task to rapidly extract relevant information from the flood of data.
webkdd/snakdd 2007: web mining and social network analysis post-workshop report. in this report, we summarize the contents and outcomes of the recent joint webkdd/snakdd 2007 workshop on web mining and social network analysis that was held in conjunction with the 13th acm sigkdd international conference on knowledge discovery and data mining (kdd 2007), august 12-15, 2007, in san jose, california. in 2007, webkdd was organized for the ninth time in a successful series of workshops on knowledge discovery on the web. the first sna-kdd workshop is created to bring together researchers who currently contribute to different aspects of social network analysis, including knowledge discovery and data mining in social network, social network modeling, multi-agent based social network simulation, complex generic network analysis. the joint 9th webkdd and 1st sna-kdd workshop was co-held with kdd 2007 and solicited papers on broad overview subject "web mining and social network analysis". we reflect on the results of the workshop, as captured in the invited talk and presentations.
making the most of your data: kdd cup 2007 "how many ratings" winner's report. we describe the ideas and methodologies that we developed in addressing the kdd cup 2007 how many ratings task, and discuss how they contributed to our success.