OpenRaiser/Intern-Atlas huggingface dataset:
Machine Learning for Encrypted Malware Traffic Classification: Accounting for Noisy Labels and Non-Stationarity
Authors: Blake Anderson, David A. McGrew
Venue: KDD
Year: 2017

The application of machine learning for the detection of malicious network traffic has been well researched over the past several decades; it is particularly appealing when the traffic is encrypted because traditional pattern-matching approaches cannot be used. Unfortunately, the promise of machine learning has been slow to materialize in the network security domain. In this paper, we highlight two primary reasons why this is the case: inaccurate ground truth and a highly non-stationary data distribution. To demonstrate and understand the effect that these pitfalls have on popular machine learning algorithms, we design and carry out experiments that show how six common algorithms perform when confronted with real network data. With our experimental results, we identify the situations in which certain classes of algorithms underperform on the task of encrypted malware traffic classification. We offer concrete recommendations for practitioners given the real-world constraints outlined. From an algorithmic perspective, we find that the random forest ensemble method outperformed competing methods. More importantly, feature engineering was decisive; we found that iterating on the initial feature set, and including features suggested by domain experts, had a much greater impact on the performance of the classification system. For example, linear regression using the more expressive feature set easily outperformed the random forest method using a standard network traffic representation on all criteria considered. Our analysis is based on millions of TLS encrypted sessions collected over 12 months from a commercial malware sandbox and two geographically distinct, large enterprise networks.
Learning Certifiably Optimal Rule Lists
Authors: Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo I. Seltzer, Cynthia Rudin
Venue: KDD
Year: 2017

We present the design and implementation of a custom discrete optimization technique for building rule lists over a categorical feature space. Our algorithm provides the optimal solution, with a certificate of optimality. By leveraging algorithmic bounds, efficient data structures, and computational reuse, we achieve several orders of magnitude speedup in time and a massive reduction of memory consumption. We demonstrate that our approach produces optimal rule lists on practical problems in seconds. This framework is a novel alternative to CART and other decision tree methods.
Improved Degree Bounds and Full Spectrum Power Laws in Preferential Attachment Networks
Authors: Chen Avin, Zvi Lotker, Yinon Nahum, David Peleg
Venue: KDD
Year: 2017

Consider a random preferential attachment model G(p) for network evolution that allows both node and edge arrivals. Starting with an arbitrary nonempty graph G0, at each time step, there are two possible events: with probability p > 0 a new node arrives and a new edge is added between the new node and an existing node, and with probability 1 - p a new edge is added between two existing nodes. In both cases, the involved existing nodes are chosen at random according to preferential attachment, i.e., with probability proportional to their degree. G(p) is known to generate power law networks, i.e., the fraction of nodes with degree k is proportional to k-β. Here β=(4-p)/(2-p) is in the range (2,3].
Unsupervised Network Discovery for Brain Imaging Data
Authors: Zilong Bai, Peter B. Walker, Anna E. Tschiffely, Fei Wang, Ian Davidson
Venue: KDD
Year: 2017

A common problem with spatiotemporal data is how to simplify the data to discover an underlying network that consists of cohesive spatial regions (nodes) and relationships between those regions (edges). This network discovery problem naturally exists in a multitude of domains including climate data (dipoles), astronomical data (gravitational lensing) and the focus of this paper, fMRI scans of human subjects. Whereas previous work requires strong supervision, we propose an unsupervised matrix tri-factorization formulation with complex constraints and spatial regularization. We show that this formulation works well in controlled experiments with synthetic networks and is able to recover the underlying ground-truth network. We then show that for real fMRI data our approach can reproduce well known results in neurology regarding the default mode network in resting-state healthy and Alzheimer affected individuals.
Planning Bike Lanes based on Sharing-Bikes' Trajectories
Authors: Jie Bao, Tianfu He, Sijie Ruan, Yanhua Li, Yu Zheng
Venue: KDD
Year: 2017

Cycling as a green transportation mode has been promoted by many governments all over the world. As a result, constructing effective bike lanes has become a crucial task for governments promoting the cycling life style, as well-planned bike paths can reduce traffic congestion and decrease safety risks for both cyclists and motor vehicle drivers. Unfortunately, existing trajectory mining approaches for bike lane planning do not consider key realistic government constraints: 1) budget limitations, 2) construction convenience, and 3) bike lane utilization.
Mining Big Data in NeuroGenetics to Understand Muscular Dystrophy
Authors: Andy Berglund
Venue: KDD
Year: 2017

The recent advances in genome sequencing and analyses of the billions of base pairs in genomic data have been a boon for moving forward our understanding of human disease. In this talk I will describe how genome sequencing has dramatically improved our understanding of the most common adult form of muscular dystrophy, which is myotonic dystrophy. Two different genetic mutations cause thousands of changes in the cells and tissues of myotonic dystrophy patients. Genome sequencing has allowed us to precisely determine the degree of changes across patients, correlate these changes to disease symptoms and allow us to determine quickly in cell and animal models the effectiveness of therapeutic strategies for myotonic dystrophy.
Industrial Machine Learning
Authors: Josh Bloom
Venue: KDD
Year: 2017

The ongoing digitization of the industrial-scale machines that power and enable human activity is itself a major global transformation. But the real revolution-in efficiencies, in improved and saved lives-will happen as machine learning automation and insights are properly coupled to the complex systems of industrial data. Leveraging a systems view of real-world use cases from aviation to transportation, I contrast the needs and approaches of consumer versus industrial machine learning. Particularly, I focus on three key areas: combining physics-based models to data-driven models, differential privacy and secure ML (including edge-to-cloud strategies), and interpretability of model predictions.
Behavior Informatics to Discover Behavior Insight for Active and Tailored Client Management
Authors: Longbing Cao
Venue: KDD
Year: 2017

Behavior is ubiquitous, and behavior intelligence and insight play an important role in data understanding and business problem-solving. Behavior Informatics [1,2] emerges as an important tool for discovering behavior intelligence and behavior insight. As a computational concept, behavior captures the aspects of the demographics of behavioral subjects and objects; social relationships or norms governing the interactions between behaviors of an individual or a group; behavior sequences or networks and their dynamics; and the impact or effect generated by the behaviors undertaken by subjects on objects. Accordingly, a behavior model [2] captures the subject and the object of a behavior or behavior sequence, the activities conducted by its subject on objects, and the relationships between activities; behavior subject, object, activities and relationships are characterized by their respective attributes. As a result, a behavior is represented as a behavior attributes-based vector; and a subject's behaviors at a time period form a vector-based sequence, namely, represented as a behavior attribute vector-based matrix [3]. With such behavior modeling and from the informatics perspective, behavior informatics takes a top-down approach to systematically and deeply represent, model, reason about, and aggregate behaviors [4]; and a bottom-up approach to analyze and learn behavior occurrences, non-occurrences, dynamics, impact, and utility [2].
Robust Top-k Multiclass SVM for Visual Category Recognition
Authors: Xiaojun Chang, Yaoliang Yu, Yi Yang
Venue: KDD
Year: 2017

Classification problems with a large number of classes inevitably involve overlapping or similar classes. In such cases it seems reasonable to allow the learning algorithm to make mistakes on similar classes, as long as the true class is still among the top-k (say) predictions. Likewise, in applications such as search engine or ad display, we are allowed to present k predictions at a time and the customer would be satisfied as long as her interested prediction is included. Inspired by the recent work of [15], we propose a very generic, robust multiclass SVM formulation that directly aims at minimizing a weighted and truncated combination of the ordered prediction scores. Our method includes many previous works as special cases. Computationally, using the Jordan decomposition Lemma we show how to rewrite our objective as the difference of two convex functions, based on which we develop an efficient algorithm that allows incorporating many popular regularizers (such as the l2 and l1 norms). We conduct extensive experiments on four real large-scale visual category recognition datasets, and obtain very promising performances.
Fast Newton Hard Thresholding Pursuit for Sparsity Constrained Nonconvex Optimization
Authors: Jinghui Chen, Quanquan Gu
Venue: KDD
Year: 2017

We propose a fast Newton hard thresholding pursuit algorithm for sparsity constrained nonconvex optimization. Our proposed algorithm reduces the per-iteration time complexity to linear in the data dimension d compared with cubic time complexity in Newton's method, while preserving faster computational and statistical convergence rates. In particular, we prove that the proposed algorithm converges to the unknown sparse model parameter at a composite rate, namely quadratic at first and linear when it gets close to the true parameter, up to the minimax optimal statistical precision of the underlying model. Thorough experiments on both synthetic and real datasets demonstrate that our algorithm outperforms the state-of-the-art optimization algorithms for sparsity constrained optimization.
TensorFlow Estimators: Managing Simplicity vs. Flexibility in High-Level Machine Learning Frameworks
Authors: Heng-Tze Cheng, Zakaria Haque, Lichan Hong, Mustafa Ispir, Clemens Mewald, Illia Polosukhin, Georgios Roumpos, D. Sculley, Jamie Smith, David Soergel, Yuan Tang, Philipp Tucker, Martin Wicke, Cassandra Xia, Jianwei Xie
Venue: KDD
Year: 2017

We present a framework for specifying, training, evaluating, and deploying machine learning models. Our focus is on simplifying cutting edge machine learning for practitioners in order to bring such technologies into production. Recognizing the fast evolution of the field of deep learning, we make no attempt to capture the design space of all possible model architectures in a domain-specific language (DSL) or similar configuration language. We allow users to write code to define their models, but provide abstractions that guide developers to write models in ways conducive to productionization. We also provide a unifying Estimator interface, making it possible to write downstream infrastructure (e.g. distributed training, hyperparameter tuning) independent of the model implementation. We balance the competing demands for flexibility and simplicity by offering APIs at different levels of abstraction, making common model architectures available out of the box, while providing a library of utilities designed to speed up experimentation with model architectures. To make out of the box models flexible and usable across a wide range of problems, these canned Estimators are parameterized not only over traditional hyperparameters, but also using feature columns, a declarative specification describing how to interpret input data We discuss our experience in using this framework in research and production environments, and show the impact on code health, maintainability, and development speed.
Unsupervised Feature Selection in Signed Social Networks
Authors: Kewei Cheng, Jundong Li, Huan Liu
Venue: KDD
Year: 2017

The rapid growth of social media services brings a large amount of high-dimensional social media data at an unprecedented rate. Feature selection is powerful to prepare high-dimensional data by finding a subset of relevant features. A vast majority of existing feature selection algorithms for social media data exclusively focus on positive interactions among linked instances such as friendships and user following relations. However, in many real-world social networks, instances may also be negatively interconnected. Recent work shows that negative links have an added value over positive links in advancing many learning tasks. In this paper, we study a novel problem of unsupervised feature selection in signed social networks and propose a novel framework SignedFS. In particular, we provide a principled way to model positive and negative links for user latent representation learning. Then we embed the user latent representations into feature selection when label information is not available. Also, we revisit the principle of homophily and balance theory in signed social networks and incorporate the signed graph regularization into the feature selection framework to capture the first-order and the second-order proximity among users in signed social networks. Experiments on two real-world signed social networks demonstrate the effectiveness of our proposed framework. Further experiments are conducted to understand the impacts of different components of SignedFS.
GRAM: Graph-based Attention Model for Healthcare Representation Learning
Authors: Edward Choi, Mohammad Taha Bahadori, Le Song, Walter F. Stewart, Jimeng Sun
Venue: KDD
Year: 2017

Deep learning methods exhibit promising performance for predictive modeling in healthcare, but two important challenges remain: - Data insufficiency: Often in healthcare predictive modeling, the sample size is insufficient for deep learning methods to achieve satisfactory results. Interpretation: The representations learned by deep learning methods should align with medical knowledge. To address these challenges, we propose GRaph-based Attention Model (GRAM) that supplements electronic health records (EHR) with hierarchical information inherent to medical ontologies. Based on the data volume and the ontology structure, GRAM represents a medical concept as a combination of its ancestors in the ontology via an attention mechanism. We compared predictive performance (i.e. accuracy, data needs, interpretability) of GRAM to various methods including the recurrent neural network (RNN) in two sequential diagnoses prediction tasks and one heart failure prediction task. Compared to the basic RNN, GRAM achieved 10% higher accuracy for predicting diseases rarely observed in the training data and 3% improved area under the ROC curve for predicting heart failure using an order of magnitude less training data. Additionally, unlike other methods, the medical concept representations learned by GRAM are well aligned with the medical ontology. Finally, GRAM exhibits intuitive attention behaviors by adaptively generalizing to higher level concepts when facing data insufficiency at the lower level concepts.
HyperLogLog Hyperextended: Sketches for Concave Sublinear Frequency Statistics
Authors: Edith Cohen
Venue: KDD
Year: 2017

One of the most common statistics computed over data elements is the number of distinct keys. A thread of research pioneered by Flajolet and Martin three decades ago culminated in the design of optimal approximate counting sketches, which have size that is double logarithmic in the number of distinct keys and provide estimates with a small relative error. Moreover, the sketches are composable, and thus suitable for streamed, parallel, or distributed computation. We consider here all statistics of the frequency distribution of keys, where a contribution of a key to the aggregate is concave and grows (sub)linearly with its frequency. These fundamental aggregations are very common in text, graphs, and logs analysis and include logarithms, low frequency moments, and cap statistics. We design composable sketches of double-logarithmic size for all concave sublinear statistics. Our design combines theoretical optimality and practical simplicity. In a nutshell, we specify tailored mapping functions of data elements to output elements so that our target statistics on the data elements is approximated by the (max-) distinct statistics of the output elements, which can be approximated using off-the-shelf sketches. Our key insight is relating these target statistics to the complement Laplace transform of the input frequencies.
Algorithmic Decision Making and the Cost of Fairness
Authors: Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, Aziz Huq
Venue: KDD
Year: 2017

Algorithms are now regularly used to decide whether defendants awaiting trial are too dangerous to be released back into the community. In some cases, black defendants are substantially more likely than white defendants to be incorrectly classified as high risk. To mitigate such disparities, several techniques have recently been proposed to achieve algorithmic fairness. Here we reformulate algorithmic fairness as constrained optimization: the objective is to maximize public safety while satisfying formal fairness constraints designed to reduce racial disparities. We show that for several past definitions of fairness, the optimal algorithms that result require detaining defendants above race-specific risk thresholds. We further show that the optimal unconstrained algorithm requires applying a single, uniform threshold to all defendants. The unconstrained algorithm thus maximizes public safety while also satisfying one important understanding of equality: that all individuals are held to the same standard, irrespective of race. Because the optimal constrained and unconstrained algorithms generally differ, there is tension between improving public safety and satisfying prevailing notions of algorithmic fairness. By examining data from Broward County, Florida, we show that this trade-off can be large in practice. We focus on algorithms for pretrial release decisions, but the principles we discuss apply to other domains, and also to human decision makers carrying out structured decision rules.
Estimation of Recent Ancestral Origins of Individuals on a Large Scale
Authors: Ross E. Curtis, Ahna Reza Girshick
Venue: KDD
Year: 2017

The last ten years have seen an exponential growth of direct-to-consumer genomics. One popular feature of these tests is the report of a distant ancestral inference profile-a breakdown of the regions of the world where the test-taker's ancestors may have lived. While current methods and products generally focus on the more distant past (e.g., thousands of years ago), we have recently demonstrated that by leveraging network analysis tools such as community detection, more recent ancestry can be identified. However, using a network analysis tool like community detection on a large network with potentially millions of nodes is not feasible in a live production environment where hundreds or thousands of new genotypes are processed every day. In this study, we describe a classification method that leverages network features to assign individuals to communities in a large network corresponding to recent ancestry. We recently launched a beta version of this research as a new product feature at AncestryDNA.
Learning Tree-Structured Detection Cascades for Heterogeneous Networks of Embedded Devices
Authors: Hamid Dadkhahi, Benjamin M. Marlin
Venue: KDD
Year: 2017

In this paper, we present a new approach to learning cascaded classifiers for use in computing environments that involve networks of heterogeneous and resource-constrained, low-power embedded compute and sensing nodes. We present a generalization of the classical linear detection cascade to the case of tree-structured cascades where different branches of the tree execute on different physical compute nodes in the network. Different nodes have access to different features, as well as access to potentially different computation and energy resources. We concentrate on the problem of jointly learning the parameters for all of the classifiers in the cascade given a fixed cascade architecture and a known set of costs required to carry out the computation at each node. To accomplish the objective of joint learning of all detectors, we propose a novel approach to combining classifier outputs during training that better matches the hard cascade setting in which the learned system will be deployed. This work is motivated by research in the area of mobile health where energy efficient real time detectors integrating information from multiple wireless on-body sensors and a smart phone are needed for real-time monitoring and the delivery of just-in-time adaptive interventions. We evaluate our framework on mobile sensor-based human activity recognition and mobile health detector learning problems.
AESOP: Automatic Policy Learning for Predicting and Mitigating Network Service Impairments
Authors: Supratim Deb, Zihui Ge, Sastry Isukapalli, Sarat C. Puthenpura, Shobha Venkataraman, He Yan, Jennifer Yates
Venue: KDD
Year: 2017

Efficient management and control of modern and next-gen networks is of paramount importance as networks have to maintain highly reliable service quality whilst supporting rapid growth in traffic demand and new application services. Rapid mitigation of network service degradations is a key factor in delivering high service quality. Automation is vital to achieving rapid mitigation of issues, particularly at the network edge where the scale and diversity is the greatest. This automation involves the rapid detection, localization and (where possible) repair of service-impacting faults and performance impairments. However, the most significant challenge here is knowing what events to detect, how to correlate events to localize an issue and what mitigation actions should be performed in response to the identified issues. These are defined as policies to systems such as ECOMP.
It Takes More than Math and Engineering to Hit the Bullseye with Data
Authors: Paritosh Desai
Venue: KDD
Year: 2017

Adopting algorithmic decision-making in a large and complex enterprise such as a Fortune 50 retailer like Target takes much more than clean, reliable data and great data mining capabilities. Yet data practitioners too often start with advanced math and fancy algorithms, rather than working hand-in-hand with business partners to identify and understand the biggest business problems. (Then teams should move onto how algorithms can be applied to those problems.) Another key step for data scientists at large organizations: ensuring that their business partners -- the merchants, marketers and supply chain experts -- have a base-line understanding of advanced models as well as the proper analytical support tools. Obtaining widespread buy-in and enthusiasm also requires providing a user-friendly interface for business partners with optionality and flexibility that allows the intelligence to be applied to the many varied issues facing a modern retailer, from personalization to supply chain transformation to decisions on assortment and pricing. This talk will explore effective practices and processes -- the do's and don'ts -- for data scientists to succeed in large, complex organizations like a retailer with 1,800+ stores, major marketing campaigns across multiple channels and a fast growing online business.