OpenRaiser/Intern-Atlas huggingface dataset:
Digesting Network Traffic for Forensic Investigation Using Digital Signal Processing Techniques
Venue: arXiv cs.NI 2019
Year: 2019

One of the most important practices of cybercrime investigations is to search a network traffic history for an excerpt of traffic, such as the disclosed information of an organization or a worm signature. In post-mortem investigations, criminals and targets are detected by attributing the excerpt to payloads of traffic flows. Since it is impossible to store the high volume of data related to long-term traffic history, payload attribution systems (PAS) based on storing a compact digest of traffic using Bloom filters have been presented in the literature. In these systems, querying the stored digest for an excerpt results in a low number of suspects instead of certain criminals. In this paper, we present a different PAS which is based on simple and widespread digital signal processing techniques. Our traffic digesting scheme has been inspired by DSP-based compression algorithms. The proposed payload attribution system, named DSPAS, not only results in a low false positive rate but also outperforms previous schemes in response to wildcard queries which are essentially applicable for complex excerpts such as the signature of polymorphic worms. Our theoretical analysis and practical evaluations show that DSPAS results in a significantly lower false positive rate and also a lower processing time for wildcard queries in comparison to previous works. Moreover, our PAS can prevent a malicious insider from evading the PAS by its ability to find strings similar to a queried excerpt.
Generative Adversarial Networks for Failure Prediction
Venue: arXiv cs.LG 2019
Year: 2019

Prognostics and Health Management (PHM) is an emerging engineering discipline which is concerned with the analysis and prediction of equipment health and performance. One of the key challenges in PHM is to accurately predict impending failures in the equipment. In recent years, solutions for failure prediction have evolved from building complex physical models to the use of machine learning algorithms that leverage the data generated by the equipment. However, failure prediction problems pose a set of unique challenges that make direct application of traditional classification and prediction algorithms impractical. These challenges include the highly imbalanced training data, the extremely high cost of collecting more failure samples, and the complexity of the failure patterns. Traditional oversampling techniques will not be able to capture such complexity and accordingly result in overfitting the training data. This paper addresses these challenges by proposing a novel algorithm for failure prediction using Generative Adversarial Networks (GAN-FP). GAN-FP first utilizes two GAN networks to simultaneously generate training samples and build an inference network that can be used to predict failures for new samples. GAN-FP first adopts an infoGAN to generate realistic failure and non-failure samples, and initialize the weights of the first few layers of the inference network. The inference network is then tuned by optimizing a weighted loss objective using only real failure and non-failure samples. The inference network is further tuned using a second GAN whose purpose is to guarantee the consistency between the generated samples and corresponding labels. GAN-FP can be used for other imbalanced classification problems as well.
GENN: Predicting Correlated Drug-drug Interactions with Graph Energy Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Gaining more comprehensive knowledge about drug-drug interactions (DDIs) is one of the most important tasks in drug development and medical practice. Recently graph neural networks have achieved great success in this task by modeling drugs as nodes and drug-drug interactions as links and casting DDI predictions as link prediction problems. However, correlations between link labels (e.g., DDI types) were rarely considered in existing works. We propose the graph energy neural network (GENN) to explicitly model link type correlations. We formulate the DDI prediction task as a structure prediction problem and introduce a new energy-based model where the energy function is defined by graph neural networks. Experiments on two real-world DDI datasets demonstrated that GENN is superior to many baselines without consideration of link type correlations and achieved $13.77\%$ and $5.01\%$ PR-AUC improvement on the two datasets, respectively. We also present a case study in which \mname can better capture meaningful DDI correlations compared with baseline models.
Privacy Preserving Stochastic Channel-Based Federated Learning with Neural Network Pruning
Venue: arXiv cs.LG 2019
Year: 2019

Artificial neural network has achieved unprecedented success in a wide variety of domains such as classifying, predicting and recognizing objects. This success depends on the availability of big data since the training process requires massive and representative data sets. However, data collection is often prevented by privacy concerns and people want to take control over their sensitive information during both training and using processes. To address this problem, we propose a privacy-preserving method for the distributed system, Stochastic Channel-Based Federated Learning (SCBF), which enables the participants to train a high-performance model cooperatively without sharing their inputs. We design, implement and evaluate a channel-based update algorithm for the central server in a distributed system, which selects the channels with regard to the most active features in a training loop and uploads them as learned information from local datasets. A pruning process is applied to the algorithm based on the validation set, which serves as a model accelerator. In the experiment, our model presents equal performances and higher saturating speed than the Federated Averaging method which reveals all the parameters of local models to the server when updating. We also demonstrate that the converging rates could be increased by introducing a pruning process.
Tensor-based algorithms for image classification
Venue: arXiv cs.LG 2019
Year: 2019

The interest in machine learning with tensor networks has been growing rapidly in recent years. We show that tensor-based methods developed for learning the governing equations of dynamical systems from data can, in the same way, be used for supervised learning problems and propose two novel approaches for image classification. One is a kernel-based reformulation of the previously introduced MANDy (multidimensional approximation of nonlinear dynamics), the other an alternating ridge regression in the tensor-train format. We apply both methods to the MNIST and fashion MNIST data set and show that the approaches are competitive with state-of-the-art neural network-based classifiers.
Recurrent neural network based decision support system
Venue: arXiv cs.NE 2019
Year: 2019

Decision Support Systems (DSS) in complex installations play a crucial role in assisting operators in decision making during abnormal transients and process disturbances, by actively displaying the status of the system and recording events, time of occurrence and suggesting relevant actions. The complexity and dynamics of complex systems require a careful selection of suitable neural network architecture, so as to improve diagnostic accuracy. In this work, we present a technique to develop a fault diagnostic decision support using recurrent neural network and Principal Component Analysis (PCA). We utilized the PCA method for noise filtering in the pre-diagnostic stage, and evaluate the predictive capability of radial basis recurrent network on a representative data derived from the simulation of a pressurized nuclear reactor. The process was validated using data from different fault scenarios, and the fault signatures were used as the input. The predictive outputs required are the location and sizes of the faults. The result shows that the radial basis network gives accurate predictions. Selected hyperparameters and diagnostic results are also presented in this paper.
A Benchmark to Evaluate InfiniBand Solutions for Java Applications
Venue: arXiv cs.NI 2019
Year: 2019

Low-latency network interconnects, such as InfiniBand, are commonly used in HPC centers and are even accessible with todays cloud providers offering equipped instances for rent. Most big data applications and frameworks are written in Java. But, the JVM environment alone does not provide interfaces to directly utilize InfiniBand networks. In this paper, we present the ``Java InfiniBand Benchmark'' to evaluate the currently available (and supported) ``low-level'' solutions to utilize InfiniBand in Java. It evaluates socket- and verbs-based libraries using typical network microbenchmarks regarding throughput and latency. Furthermore, we present evaluation results of the solutions on two hardware configurations with 56 Gbit/s and 100 Gbit/s InfiniBand NICs. With transparency often traded for performance and vice versa, the benchmark helps developers with studying the pros and cons of each solution and support them in their decision which solution is more suitable for their existing or new use-case.
Characterizing Membership Privacy in Stochastic Gradient Langevin Dynamics
Venue: arXiv cs.LG 2019
Year: 2019

Bayesian deep learning is recently regarded as an intrinsic way to characterize the weight uncertainty of deep neural networks~(DNNs). Stochastic Gradient Langevin Dynamics~(SGLD) is an effective method to enable Bayesian deep learning on large-scale datasets. Previous theoretical studies have shown various appealing properties of SGLD, ranging from the convergence properties to the generalization bounds. In this paper, we study the properties of SGLD from a novel perspective of membership privacy protection (i.e., preventing the membership attack). The membership attack, which aims to determine whether a specific sample is used for training a given DNN model, has emerged as a common threat against deep learning algorithms. To this end, we build a theoretical framework to analyze the information leakage (w.r.t. the training dataset) of a model trained using SGLD. Based on this framework, we demonstrate that SGLD can prevent the information leakage of the training dataset to a certain extent. Moreover, our theoretical analysis can be naturally extended to other types of Stochastic Gradient Markov Chain Monte Carlo (SG-MCMC) methods. Empirical results on different datasets and models verify our theoretical findings and suggest that the SGLD algorithm can not only reduce the information leakage but also improve the generalization ability of the DNN models in real-world applications.
Parallelizing Training of Deep Generative Models on Massive Scientific Datasets
Venue: arXiv cs.DC 2019
Year: 2019

Training deep neural networks on large scientific data is a challenging task that requires enormous compute power, especially if no pre-trained models exist to initialize the process. We present a novel tournament method to train traditional as well as generative adversarial networks built on LBANN, a scalable deep learning framework optimized for HPC systems. LBANN combines multiple levels of parallelism and exploits some of the worlds largest supercomputers. We demonstrate our framework by creating a complex predictive model based on multi-variate data from high-energy-density physics containing hundreds of millions of images and hundreds of millions of scalar values derived from tens of millions of simulations of inertial confinement fusion. Our approach combines an HPC workflow and extends LBANN with optimized data ingestion and the new tournament-style training algorithm to produce a scalable neural network architecture using a CORAL-class supercomputer. Experimental results show that 64 trainers (1024 GPUs) achieve a speedup of 70.2 over a single trainer (16 GPUs) baseline, and an effective 109% parallel efficiency.
Few-shot tweet detection in emerging disaster events
Venue: arXiv cs.LG 2019
Year: 2019

Social media sources can provide crucial information in crisis situations, but discovering relevant messages is not trivial. Methods have so far focused on universal detection models for all kinds of crises or for certain crisis types (e.g. floods). Event-specific models could implement a more focused search area, but collecting data and training new models for a crisis that is already in progress is costly and may take too much time for a prompt response. As a compromise, manually collecting a small amount of example messages is feasible. Few-shot models can generalize to unseen classes with such a small handful of examples, and do not need be trained anew for each event. We compare how few-shot approaches (matching networks and prototypical networks) perform for this task. Since this is essentially a one-class problem, we also demonstrate how a modified one-class version of prototypical models can be used for this application.
Change Detection in Noisy Dynamic Networks: A Spectral Embedding Approach
Venue: arXiv cs.LG 2019
Year: 2019

Change detection in dynamic networks is an important problem in many areas, such as fraud detection, cyber intrusion detection and health care monitoring. It is a challenging problem because it involves a time sequence of graphs, each of which is usually very large and sparse with heterogeneous vertex degrees, resulting in a complex, high dimensional mathematical object. Spectral embedding methods provide an effective way to transform a graph to a lower dimensional latent Euclidean space that preserves the underlying structure of the network. Although change detection methods that use spectral embedding are available, they do not address sparsity and degree heterogeneity that usually occur in noisy real-world graphs and a majority of these methods focus on changes in the behaviour of the overall network. In this paper, we adapt previously developed techniques in spectral graph theory and propose a novel concept of applying Procrustes techniques to embedded points for vertices in a graph to detect changes in entity behaviour. Our spectral embedding approach not only addresses sparsity and degree heterogeneity issues, but also obtains an estimate of the appropriate embedding dimension. We call this method CDP (change detection using Procrustes analysis). We demonstrate the performance of CDP through extensive simulation experiments and a real-world application. CDP successfully detects various types of vertex-based changes including (i) changes in vertex degree, (ii) changes in community membership of vertices, and (iii) unusual increase or decrease in edge weight between vertices. The change detection performance of CDP is compared with two other baseline methods that employ alternative spectral embedding approaches. In both cases, CDP generally shows superior performance.
Carrots or Sticks? The Effectiveness of Subsidies and Tolls in Congestion Games
Venue: arXiv cs.GT 2019
Year: 2019

Are rewards or penalties more effective in influencing user behavior? This work compares the effectiveness of subsidies and tolls in incentivizing users in congestion games. The predominantly studied method of influencing user behavior in network routing problems is to institute taxes which alter users' observed costs in a manner that causes their self-interested choices to more closely align with a system-level objective. Another feasible method to accomplish the same goal is to subsidize the users' actions that are preferable from a system-level perspective. We show that, when users behave similarly and predictably, subsidies offer comparable performance guarantees to tolls while requiring smaller monetary transactions with users; however, in the presence of unknown player heterogeneity, subsidies fail to offer the same performance as tolls. We further investigate these relationships in affine congestion games, deriving explicit performance bounds under optimal tolls and subsidies with and without user heterogeneity; we show that the differences in performance can be significant.
Biased Aggregation, Rollout, and Enhanced Policy Improvement for Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

We propose a new aggregation framework for approximate dynamic programming, which provides a connection with rollout algorithms, approximate policy iteration, and other single and multistep lookahead methods. The central novel characteristic is the use of a bias function $V$ of the state, which biases the values of the aggregate cost function towards their correct levels. The classical aggregation framework is obtained when $V\equiv0$, but our scheme works best when $V$ is a known reasonably good approximation to the optimal cost function $J^*$. When $V$ is equal to the cost function $J_μ$ of some known policy $μ$ and there is only one aggregate state, our scheme is equivalent to the rollout algorithm based on $μ$ (i.e., the result of a single policy improvement starting with the policy $μ$). When $V=J_μ$ and there are multiple aggregate states, our aggregation approach can be used as a more powerful form of improvement of $μ$. Thus, when combined with an approximate policy evaluation scheme, our approach can form the basis for a new and enhanced form of approximate policy iteration. When $V$ is a generic bias function, our scheme is equivalent to approximation in value space with lookahead function equal to $V$ plus a local correction within each aggregate state. The local correction levels are obtained by solving a low-dimensional aggregate DP problem, yielding an arbitrarily close approximation to $J^*$, when the number of aggregate states is sufficiently large. Except for the bias function, the aggregate DP problem is similar to the one of the classical aggregation framework, and its algorithmic solution by simulation or other methods is nearly identical to one for classical aggregation, assuming values of $V$ are available when needed.
Modal Logic via Global Consequence
Venue: arXiv cs.LO 2019
Year: 2019

In modal logic, semantic consequence is usually defined locally by truth preservation at all worlds in all models (with respect to a class of frames). It can also be defined globally by truth preservation in all models (with respect to a class of frames). The latter is called global consequence, which is much less studied than the standard local one. In this paper we first study the relationship between local and global consequence. Then we give some correspondence results for global consequence. Finally, we illustrate two applications of global consequence, connecting it with informational consequence and update consequence proposed in formal semantics. Some results in the paper are already known, which are collected in the paper for the sake of completeness. The others appear to be new. We suggest that global consequence is not only interesting theoretically, but also useful for application.
Risk-Aware Reasoning for Autonomous Vehicles
Venue: arXiv cs.AI 2019
Year: 2019

A significant barrier to deploying autonomous vehicles (AVs) on a massive scale is safety assurance. Several technical challenges arise due to the uncertain environment in which AVs operate such as road and weather conditions, errors in perception and sensory data, and also model inaccuracy. In this paper, we propose a system architecture for risk-aware AVs capable of reasoning about uncertainty and deliberately bounding the risk of collision below a given threshold. We discuss key challenges in the area, highlight recent research developments, and propose future research directions in three subsystems. First, a perception subsystem that detects objects within a scene while quantifying the uncertainty that arises from different sensing and communication modalities. Second, an intention recognition subsystem that predicts the driving-style and the intention of agent vehicles (and pedestrians). Third, a planning subsystem that takes into account the uncertainty, from perception and intention recognition subsystems, and propagates all the way to control policies that explicitly bound the risk of collision. We believe that such a white-box approach is crucial for future adoption of AVs on a large scale.
Approximation algorithms for maximally balanced connected graph partition
Venue: arXiv cs.DS 2019
Year: 2019

Given a simple connected graph $G = (V, E)$, we seek to partition the vertex set $V$ into $k$ non-empty parts such that the subgraph induced by each part is connected, and the partition is maximally balanced in the way that the maximum cardinality of these $k$ parts is minimized. We refer this problem to as {\em min-max balanced connected graph partition} into $k$ parts and denote it as {\sc $k$-BGP}. The general vertex-weighted version of this problem on trees has been studied since about four decades ago, which admits a linear time exact algorithm; the vertex-weighted {\sc $2$-BGP} and {\sc $3$-BGP} admit a $5/4$-approximation and a $3/2$-approximation, respectively; but no approximability result exists for {\sc $k$-BGP} when $k \ge 4$, except a trivial $k$-approximation. In this paper, we present another $3/2$-approximation for our cardinality {\sc $3$-BGP} and then extend it to become a $k/2$-approximation for {\sc $k$-BGP}, for any constant $k \ge 3$. Furthermore, for {\sc $4$-BGP}, we propose an improved $24/13$-approximation. To these purposes, we have designed several local improvement operations, which could be useful for related graph partition problems.
Optimising energy and overhead for large parameter space simulations
Venue: arXiv cs.PF 2019
Year: 2019

Many systems require optimisation over multiple objectives, where objectives are characteristics of the system such as energy consumed or increase in time to perform the work. Optimisation is performed by selecting the `best' set of input parameters to elicit the desired objectives. However, the parameter search space can often be far larger than can be searched in a reasonable time. Additionally, the objectives are often mutually exclusive -- leading to a decision being made as to which objective is more important or optimising over a combination of the objectives. This work is an application of a Genetic Algorithm to identify the Pareto frontier for finding the optimal parameter sets for all combinations of objectives. A Pareto frontier can be used to identify the sets of optimal parameters for which each is the `best' for a given combination of objectives -- thus allowing decisions to be made with full knowledge. We demonstrate this approach for the HTC-Sim simulation system in the case where a Reinforcement Learning scheduler is tuned for the two objectives of energy consumption and task overhead. Demonstrating that this approach can reduce the energy consumed by ~36% over previously published work without significantly increasing the overhead.