OpenRaiser/Intern-Atlas huggingface dataset:
AMSI-Based Detection of Malicious PowerShell Code Using Contextual Embeddings
Venue: arXiv cs.CR 2019
Year: 2019

PowerShell is a command-line shell, supporting a scripting language. It is widely used in organizations for configuration management and task automation but is also increasingly used by cybercriminals for launching cyberattacks against organizations, mainly because it is pre-installed on Windows machines and exposes strong functionality that may be leveraged by attackers. This makes the problem of detecting malicious PowerShell code both urgent and challenging. Microsoft's Antimalware Scan Interface (AMSI) allows defending systems to scan all the code passed to scripting engines such as PowerShell prior to its execution. In this work, we conduct the first study of malicious PowerShell code detection using the information made available by AMSI. We present several novel deep-learning based detectors of malicious PowerShell code that employ pretrained contextual embeddings of words from the PowerShell "language". A known problem in the cybersecurity domain is that labeled data is relatively scarce in comparison with unlabeled data, making it difficult to devise effective supervised detection of malicious activity of many types. This is also the case with PowerShell code. Our work shows that this problem can be mitigated by learning a pretrained contextual embedding based on unlabeled data. We trained and evaluated our models using real-world data, collected using AMSI from a large antimalware vendor. Our performance analysis establishes that the use of unlabeled data for the embedding significantly improved the performance of our detectors. Our best-performing model uses an architecture that enables the processing of textual signals from both the character and token levels and obtains a true positive rate of nearly 90% while maintaining a low false-positive rate of less than 0.1%.
Improving Neural Networks by Adopting Amplifying and Attenuating Neurons
Venue: arXiv cs.NE 2019
Year: 2019

In the present study, an amplifying neuron and attenuating neuron, which can be easily implemented into neural networks without any significant additional computational effort, are proposed. The activated output value is squared for the amplifying neuron, while the value becomes its reciprocal for the attenuating one. Theoretically, the order of neural networks increases when the amplifying neuron is placed in the hidden layer. The performance assessments of neural networks were conducted to verify that the amplifying and attenuating neurons enhance the performance of neural networks. From the numerical experiments, it was revealed that the neural networks that contain the amplifying and attenuating neurons yield more accurate results, compared to those without them.
Beyond Cookie Monster Amnesia: Real World Persistent Online Tracking
Venue: arXiv cs.CR 2019
Year: 2019

Browser fingerprinting is a relatively new method of uniquely identifying browsers that can be used to track web users. In some ways it is more privacy-threatening than tracking via cookies, as users have no direct control over it. A number of authors have considered the wide variety of techniques that can be used to fingerprint browsers; however, relatively little information is available on how widespread browser fingerprinting is, and what information is collected to create these fingerprints in the real world. To help address this gap, we crawled the 10,000 most popular websites; this gave insights into the number of websites that are using the technique, which websites are collecting fingerprinting information, and exactly what information is being retrieved. We found that approximately 69\% of websites are, potentially, involved in first-party or third-party browser fingerprinting. We further found that third-party browser fingerprinting, which is potentially more privacy-damaging, appears to be predominant in practice. We also describe \textit{FingerprintAlert}, a freely available browser extension we developed that detects and, optionally, blocks fingerprinting attempts by visited websites.
CUDA-Self-Organizing feature map based visual sentiment analysis of bank customer complaints for Analytical CRM
Venue: arXiv cs.NE 2019
Year: 2019

With the widespread use of social media, companies now have access to a wealth of customer feedback data which has valuable applications to Customer Relationship Management (CRM). Analyzing customer grievances data, is paramount as their speedy non-redressal would lead to customer churn resulting in lower profitability. In this paper, we propose a descriptive analytics framework using Self-organizing feature map (SOM), for Visual Sentiment Analysis of customer complaints. The network learns the inherent grouping of the complaints automatically which can then be visualized too using various techniques. Analytical Customer Relationship Management (ACRM) executives can draw useful business insights from the maps and take timely remedial action. We also propose a high-performance version of the algorithm CUDASOM (CUDA based Self Organizing feature Map) implemented using NVIDIA parallel computing platform, CUDA, which speeds up the processing of high-dimensional text data and generates fast results. The efficacy of the proposed model has been demonstrated on the customer complaints data regarding the products and services of four leading Indian banks. CUDASOM achieved an average speed up of 44 times. Our approach can expand research into intelligent grievance redressal system to provide rapid solutions to the complaining customers.
Unraveling the Veil of Subspace RIP Through Near-Isometry on Subspaces
Venue: arXiv cs.IT 2019
Year: 2019

Dimensionality reduction is a popular approach to tackle high-dimensional data with low-dimensional nature. Subspace Restricted Isometry Property, a newly-proposed concept, has proved to be a useful tool in analyzing the effect of dimensionality reduction algorithms on subspaces. In this paper, we provide a characterization of subspace Restricted Isometry Property, asserting that matrices which act as a near-isometry on low-dimensional subspaces possess subspace Restricted Isometry Property. This points out a unified approach to discuss subspace Restricted Isometry Property. Its power is further demonstrated by the possibility to prove with this result the subspace RIP for a large variety of random matrices encountered in theory and practice, including subgaussian matrices, partial Fourier matrices, partial Hadamard matrices, partial circulant/Toeplitz matrices, matrices with independent strongly regular rows (for instance, matrices with independent entries having uniformly bounded $4+ε$ moments), and log-concave ensembles. Thus our result could extend the applicability of random projections in subspace-based machine learning algorithms including subspace clustering and allow for the application of some useful random matrices which are easier to implement on hardware or are more efficient to compute.
Automatic Generation of Level Maps with the Do What's Possible Representation
Venue: arXiv cs.AI 2019
Year: 2019

Automatic generation of level maps is a popular form of automatic content generation. In this study, a recently developed technique employing the {\em do what's possible} representation is used to create open-ended level maps. Generation of the map can continue indefinitely, yielding a highly scalable representation. A parameter study is performed to find good parameters for the evolutionary algorithm used to locate high-quality map generators. Variations on the technique are presented, demonstrating its versatility, and an algorithmic variant is given that both improves performance and changes the character of maps located. The ability of the map to adapt to different regions where the map is permitted to occupy space are also tested.
From semantics to execution: Integrating action planning with reinforcement learning for robotic causal problem-solving
Venue: arXiv cs.LG 2019
Year: 2019

Reinforcement learning is an appropriate and successful method to robustly perform low-level robot control under noisy conditions. Symbolic action planning is useful to resolve causal dependencies and to break a causally complex problem down into a sequence of simpler high-level actions. A problem with the integration of both approaches is that action planning is based on discrete high-level action- and state spaces, whereas reinforcement learning is usually driven by a continuous reward function. However, recent advances in reinforcement learning, specifically, universal value function approximators and hindsight experience replay, have focused on goal-independent methods based on sparse rewards. In this article, we build on these novel methods to facilitate the integration of action planning with reinforcement learning by exploiting the reward-sparsity as a bridge between the high-level and low-level state- and control spaces. As a result, we demonstrate that the integrated neuro-symbolic method is able to solve object manipulation problems that involve tool use and non-trivial causal dependencies under noisy conditions, exploiting both data and knowledge.
Verification and Synthesis of Symmetric Uni-Rings for Leads-To Properties
Venue: arXiv cs.DC 2019
Year: 2019

This paper investigates the verification and synthesis of parameterized protocols that satisfy leadsto properties $R \leadsto Q$ on symmetric unidirectional rings (a.k.a. uni-rings) of deterministic and constant-space processes under no fairness and interleaving semantics, where $R$ and $Q$ are global state predicates. First, we show that verifying $R \leadsto Q$ for parameterized protocols on symmetric uni-rings is undecidable, even for deterministic and constant-space processes, and conjunctive state predicates. Then, we show that surprisingly synthesizing symmetric uni-ring protocols that satisfy $R \leadsto Q$ is actually decidable. We identify necessary and sufficient conditions for the decidability of synthesis based on which we devise a sound and complete polynomial-time algorithm that takes the predicates $R$ and $Q$, and automatically generates a parameterized protocol that satisfies $R \leadsto Q$ for unbounded (but finite) ring sizes. Moreover, we present some decidability results for cases where leadsto is required from multiple distinct $R$ predicates to different $Q$ predicates. To demonstrate the practicality of our synthesis method, we synthesize some parameterized protocols, including agreement and parity protocols.
Multi-Service Mobile Traffic Forecasting via Convolutional Long Short-Term Memories
Venue: arXiv cs.LG 2019
Year: 2019

Network slicing is increasingly used to partition network infrastructure between different mobile services. Precise service-wise mobile traffic forecasting becomes essential in this context, as mobile operators seek to pre-allocate resources to each slice in advance, to meet the distinct requirements of individual services. This paper attacks the problem of multi-service mobile traffic forecasting using a sequence-to-sequence (S2S) learning paradigm and convolutional long short-term memories (ConvLSTMs). The proposed architecture is designed so as to effectively extract complex spatiotemporal features of mobile network traffic and predict with high accuracy the future demands for individual services at city scale. We conduct experiments on a mobile traffic dataset collected in a large European metropolis, demonstrating that the proposed S2S-ConvLSTM can forecast the mobile traffic volume produced by tens of different services in advance of up to one hour, by just using measurements taken during the past hour. In particular, our solution achieves mean absolute errors (MAE) at antenna level that are below 13KBps, outperforming other deep learning approaches by up to 31.2%.
Nature-Inspired Computational Model of Population Desegregation under Group Leaders Influence
Venue: arXiv cs.MA 2019
Year: 2019

This paper presents an agent-based model of population desegregation and provides a thorough analysis of the social behavior leading to it, namely, the contact hypothesis. Based on the parameters of frequency and intensity of influence of group leaders on the population, the proposed model is constituted by two layers: 1) a physical layer of the population that is influenced by and 2) a virtual layer of group leaders. The model of negotiation and survival of group leaders are governed by the nature-inspired evolutionary process of queen ants, also known as Foundress Dilemma. The motivation of using a virtual grouping concept (instead of taking a subset of population as the group leaders) is to stay focused on finding the conditions leading individuals in a society tolerating a significantly diversified (desegregated) neighborhood, rather than, indulging into complex details, which would be more relevant to studies targeting the evolution of societal group and leaders. A geographic information system-driven simulation is performed, which reveals that: 1) desegregation is directly proportional to the frequency of group leaders' contact with the population and 2) mostly, it remains ineffective with an increase in the intensity of group leaders' contact with the population. The mechanism of group selection (the conflict resolution model resolving the Foundress Dilemma) reveals an exciting result concerning negative influence of cooperative group leaders. Most of the time, desegregation decreases with increase in cooperative leaders (the leaders enforcing desegregation) when compared with fierce leaders (the leaders enforcing segregation).
Efficient Reduction in Shape Parameter Space Dimension for Ship Propeller Blade Design
Venue: arXiv cs.CE 2019
Year: 2019

In this work, we present the results of a ship propeller design optimization campaign carried out in the framework of the research project PRELICA, funded by the Friuli Venezia Giulia regional government. The main idea of this work is to operate on a multidisciplinary level to identify propeller shapes that lead to reduced tip vortex-induced pressure and increased efficiency without altering the thrust. First, a specific tool for the bottom-up construction of parameterized propeller blade geometries has been developed. The algorithm proposed operates with a user defined number of arbitrary shaped or NACA airfoil sections, and employs arbitrary degree NURBS to represent the chord, pitch, skew and rake distribution as a function of the blade radial coordinate. The control points of such curves have been modified to generate, in a fully automated way, a family of blade geometries depending on as many as 20 shape parameters. Such geometries have then been used to carry out potential flow simulations with the Boundary Element Method based software PROCAL. Given the high number of parameters considered, such a preliminary stage allowed for a fast evaluation of the performance of several hundreds of shapes. In addition, the data obtained from the potential flow simulation allowed for the application of a parameter space reduction methodology based on active subspaces (AS) property, which suggested that the main propeller performance indices are, at a first but rather accurate approximation, only depending on a single parameter which is a linear combination of all the original geometric ones. AS analysis has also been used to carry out a constrained optimization exploiting response surface method in the reduced parameter space, and a sensitivity analysis based on such surrogate model. The few selected shapes were finally used to set up high fidelity RANS simulations and select an optimal shape.
Scaling Up Collaborative Filtering Data Sets through Randomized Fractal Expansions
Venue: arXiv cs.LG 2019
Year: 2019

Recommender system research suffers from a disconnect between the size of academic data sets and the scale of industrial production systems. In order to bridge that gap, we propose to generate large-scale user/item interaction data sets by expanding pre-existing public data sets. Our key contribution is a technique that expands user/item incidence matrices matrices to large numbers of rows (users), columns (items), and non-zero values (interactions). The proposed method adapts Kronecker Graph Theory to preserve key higher order statistical properties such as the fat-tailed distribution of user engagements, item popularity, and singular value spectra of user/item interaction matrices. Preserving such properties is key to building large realistic synthetic data sets which in turn can be employed reliably to benchmark recommender systems and the systems employed to train them. We further apply our stochastic expansion algorithm to the binarized MovieLens 20M data set, which comprises 20M interactions between 27K movies and 138K users. The resulting expanded data set has 1.2B ratings, 2.2M users, and 855K items, which can be scaled up or down.
Contextual Bandit Learning for Machine Type Communications in the Null Space of Multi-Antenna Systems
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, a novel approach based on the concept of opportunistic spatial orthogonalization (OSO) is proposed for interference management between machine type communications (MTC) and conventional cellular communications. In particular, a cellular system is considered with a multi-antenna BS in which a receive beamformer is designed to maximize the rate of a cellular user, and, a machine type aggregator (MTA) that receives data from a large set of MTDs. If there is a large number of MTDs to choose from for transmission at each time for each beamformer, one MTD can be selected such that it causes almost no interference on the BS. A comprehensive analytical study of the characteristics of such interference from several MTDs on the same beamformer is carried out. It is proven that, for each beamformer, an MTD exists such that the interference on the BS is negligible. However, the optimal implementation of OSO requires the CSI of all the links in the BS, which is not practical for MTC. To solve this problem, an online learning method based on the concept of contextual multi-armed bandits (MAB) learning is proposed. Simulation results show that is possible to implement OSO with no CSI from MTDs to the BS.
A COLD Approach to Generating Optimal Samples
Venue: arXiv cs.LG 2019
Year: 2019

Optimising discrete data for a desired characteristic using gradient-based methods involves projecting the data into a continuous latent space and carrying out optimisation in this space. Carrying out global optimisation is difficult as optimisers are likely to follow gradients into regions of the latent space that the model has not been exposed to during training; samples generated from these regions are likely to be too dissimilar to the training data to be useful. We propose Constrained Optimisation with Latent Distributions (COLD), a constrained global optimisation procedure to find samples with high values of a desired property that are similar to yet distinct from the training data. We find that on MNIST, our procedure yields optima for each of three different objectives, and that enforcing tighter constraints improves the quality and increases the diversity of the generated images. On the ChEMBL molecular dataset, our method generates a diverse set of new molecules with drug-likeness scores similar to those of the highest-scoring molecules in the training data. We also demonstrate a computationally efficient way to approximate the constraint when evaluating it exactly is computationally expensive.
PHom-GeM: Persistent Homology for Generative Models
Venue: arXiv cs.LG 2019
Year: 2019

Generative neural network models, including Generative Adversarial Network (GAN) and Auto-Encoders (AE), are among the most popular neural network models to generate adversarial data. The GAN model is composed of a generator that produces synthetic data and of a discriminator that discriminates between the generator's output and the true data. AE consist of an encoder which maps the model distribution to a latent manifold and of a decoder which maps the latent manifold to a reconstructed distribution. However, generative models are known to provoke chaotically scattered reconstructed distribution during their training, and consequently, incomplete generated adversarial distributions. Current distance measures fail to address this problem because they are not able to acknowledge the shape of the data manifold, i.e. its topological features, and the scale at which the manifold should be analyzed. We propose Persistent Homology for Generative Models, PHom-GeM, a new methodology to assess and measure the distribution of a generative model. PHom-GeM minimizes an objective function between the true and the reconstructed distributions and uses persistent homology, the study of the topological features of a space at different spatial resolutions, to compare the nature of the true and the generated distributions. Our experiments underline the potential of persistent homology for Wasserstein GAN in comparison to Wasserstein AE and Variational AE. The experiments are conducted on a real-world data set particularly challenging for traditional distance measures and generative neural network models. PHom-GeM is the first methodology to propose a topological distance measure, the bottleneck distance, for generative models used to compare adversarial samples in the context of credit card transactions.
Neural ODEs with stochastic vector field mixtures
Venue: arXiv cs.LG 2019
Year: 2019

It was recently shown that neural ordinary differential equation models cannot solve fundamental and seemingly straightforward tasks even with high-capacity vector field representations. This paper introduces two other fundamental tasks to the set that baseline methods cannot solve, and proposes mixtures of stochastic vector fields as a model class that is capable of solving these essential problems. Dynamic vector field selection is of critical importance for our model, and our approach is to propagate component uncertainty over the integration interval with a technique based on forward filtering. We also formalise several loss functions that encourage desirable properties on the trajectory paths, and of particular interest are those that directly encourage fewer expected function evaluations. Experimentally, we demonstrate that our model class is capable of capturing the natural dynamics of human behaviour; a notoriously volatile application area. Baseline approaches cannot adequately model this problem.
PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication
Venue: arXiv cs.LG 2019
Year: 2019

We develop model free PAC performance guarantees for multiple concurrent MDPs, extending recent works where a single learner interacts with multiple non-interacting agents in a noise free environment. Our framework allows noisy and resource limited communication between agents, and develops novel PAC guarantees in this extended setting. By allowing communication between the agents themselves, we suggest improved PAC-exploration algorithms that can overcome the communication noise and lead to improved sample complexity bounds. We provide a theoretically motivated algorithm that optimally combines information from the resource limited agents, thereby analyzing the interaction between noise and communication constraints that are ubiquitous in real-world systems. We present empirical results for a simple task that supports our theoretical formulations and improve upon naive information fusion methods.