OpenRaiser/Intern-Atlas huggingface dataset:
Detecting Botnet Attacks in IoT Environments: An Optimized Machine Learning Approach
Venue: arXiv cs.CR 2020
Year: 2020

The increased reliance on the Internet and the corresponding surge in connectivity demand has led to a significant growth in Internet-of-Things (IoT) devices. The continued deployment of IoT devices has in turn led to an increase in network attacks due to the larger number of potential attack surfaces as illustrated by the recent reports that IoT malware attacks increased by 215.7% from 10.3 million in 2017 to 32.7 million in 2018. This illustrates the increased vulnerability and susceptibility of IoT devices and networks. Therefore, there is a need for proper effective and efficient attack detection and mitigation techniques in such environments. Machine learning (ML) has emerged as one potential solution due to the abundance of data generated and available for IoT devices and networks. Hence, they have significant potential to be adopted for intrusion detection for IoT environments. To that end, this paper proposes an optimized ML-based framework consisting of a combination of Bayesian optimization Gaussian Process (BO-GP) algorithm and decision tree (DT) classification model to detect attacks on IoT devices in an effective and efficient manner. The performance of the proposed framework is evaluated using the Bot-IoT-2018 dataset. Experimental results show that the proposed optimized framework has a high detection accuracy, precision, recall, and F-score, highlighting its effectiveness and robustness for the detection of botnet attacks in IoT environments.
Cognitive Computing in Data-centric Paradigm
Venue: arXiv cs.AR 2020
Year: 2020

Knowledge is the most precious asset of humankind. People extract the experience from the data that provide for us the reality through the feelings. Generally speaking, it is possible to see the analogy of knowledge elaboration between humankind's way and the artificial system's way. Digital data are the "feelings" of an artificial system, and it needs to invent a method of extraction of knowledge from the Universe of data. The cognitive computing paradigm implies that a system should be able to extract the knowledge from raw data without any human-made algorithm. The first step of the paradigm is analysis of raw data streams through the discovery of repeatable patterns of data. The knowledge of relationships among the patterns provides a way to see the structures and to generalize the concepts with the goal to synthesize new statements. The cognitive computing paradigm is capable of mimicking the human's ability to generalize the notions. It is possible to say that the generalization step provides the basis for discovering the abstract notions, revealing the abstract relations of patterns and general rules of structure synthesis. If anyone continues the process of structure generalization, then it is possible to build the multi-level hierarchy of abstract notions. Moreover, discovering the generalized classes of notions is the first step towards a paradigm of artificial analytical thinking. The most critical possible responsibility of cognitive computing could be the classification of data and recognition of input data stream's states. The synthesis of new statements creates the foundation for the foreseeing the possible data states and elaboration of knowledge about new data classes by employing synthesis and checking the hypothesis.
Sub-Linear Memory: How to Make Performers SLiM
Venue: arXiv cs.LG 2020
Year: 2020

The Transformer architecture has revolutionized deep learning on sequential data, becoming ubiquitous in state-of-the-art solutions for a wide variety of applications. Yet vanilla Transformers are notoriously resource-expensive, requiring $O(L^2)$ in serial time and memory as functions of input length $L$. Recent works proposed various linear self-attention mechanisms, scaling only as $O(L)$ for serial computation. We perform a thorough analysis of recent Transformer mechanisms with linear self-attention, Performers, in terms of overall computational complexity. We observe a remarkable computational flexibility: forward and backward propagation can be performed with no approximations using sublinear memory as a function of $L$ (in addition to negligible storage for the input sequence), at a cost of greater time complexity in the parallel setting. In the extreme case, a Performer consumes only $O(1)$ memory during training, and still requires $O(L)$ time. This discovered time-memory tradeoff can be used for training or, due to complete backward-compatibility, for fine-tuning on a low-memory device, e.g. a smartphone or an earlier-generation GPU, thus contributing towards decentralized and democratized deep learning.
AC2 -- Towards Understanding Architectural Changes in Rapid Releases
Venue: arXiv cs.SE 2020
Year: 2020

Open source projects are adopting faster release cycles that reflect various changes. Therefore, comprehending the effects of these changes on software's architecture over the releases becomes necessary. However, it is challenging to keep architecture in-check and add new changes simultaneously for every release. To this end, we propose a visualization tool called AC2, which allows its users to examine the alterations in the architecture at both higher and lower levels of abstraction for the python projects. AC2 uses call graphs and collaboration graphs to show the interaction between different architectural components. The tool provides four different views to see the architectural changes. The user can examine two releases at a time to comprehend the architectural changes between the releases. AC2 can support the maintainers and developers to observe changes in the project and its influence on the architecture, which allow them to see its increasing complexity over the releases at the component level. AC2 can be downloaded at https://github.com/dheerajrox/AC2 and its demo can be seen at the website https://dheerajrox.github.io/AC2doc or on youtube https://www.youtube.com/watch?v=GNrJfZ0RCVI
Genetic Adversarial Training of Decision Trees
Venue: arXiv cs.LG 2020
Year: 2020

We put forward a novel learning methodology for ensembles of decision trees based on a genetic algorithm which is able to train a decision tree for maximizing both its accuracy and its robustness to adversarial perturbations. This learning algorithm internally leverages a complete formal verification technique for robustness properties of decision trees based on abstract interpretation, a well known static program analysis technique. We implemented this genetic adversarial training algorithm in a tool called Meta-Silvae (MS) and we experimentally evaluated it on some reference datasets used in adversarial training. The experimental results show that MS is able to train robust models that compete with and often improve on the current state-of-the-art of adversarial training of decision trees while being much more compact and therefore interpretable and efficient tree models.
Unsupervised Anomaly Detectors to Detect Intrusions in the Current Threat Landscape
Venue: arXiv cs.LG 2020
Year: 2020

Anomaly detection aims at identifying unexpected fluctuations in the expected behavior of a given system. It is acknowledged as a reliable answer to the identification of zero-day attacks to such extent, several ML algorithms that suit for binary classification have been proposed throughout years. However, the experimental comparison of a wide pool of unsupervised algorithms for anomaly-based intrusion detection against a comprehensive set of attacks datasets was not investigated yet. To fill such gap, we exercise seventeen unsupervised anomaly detection algorithms on eleven attack datasets. Results allow elaborating on a wide range of arguments, from the behavior of the individual algorithm to the suitability of the datasets to anomaly detection. We conclude that algorithms as Isolation Forests, One-Class Support Vector Machines and Self-Organizing Maps are more effective than their counterparts for intrusion detection, while clustering algorithms represent a good alternative due to their low computational complexity. Further, we detail how attacks with unstable, distributed or non-repeatable behavior as Fuzzing, Worms and Botnets are more difficult to detect. Ultimately, we digress on capabilities of algorithms in detecting anomalies generated by a wide pool of unknown attacks, showing that achieved metric scores do not vary with respect to identifying single attacks.
Blurring Fools the Network -- Adversarial Attacks by Feature Peak Suppression and Gaussian Blurring
Venue: arXiv cs.CV 2020
Year: 2020

Existing pixel-level adversarial attacks on neural networks may be deficient in real scenarios, since pixel-level changes on the data cannot be fully delivered to the neural network after camera capture and multiple image preprocessing steps. In contrast, in this paper, we argue from another perspective that gaussian blurring, a common technique of image preprocessing, can be aggressive itself in specific occasions, thus exposing the network to real-world adversarial attacks. We first propose an adversarial attack demo named peak suppression (PS) by suppressing the values of peak elements in the features of the data. Based on the blurring spirit of PS, we further apply gaussian blurring to the data, to investigate the potential influence and threats of gaussian blurring to performance of the network. Experiment results show that PS and well-designed gaussian blurring can form adversarial attacks that completely change classification results of a well-trained target network. With the strong physical significance and wide applications of gaussian blurring, the proposed approach will also be capable of conducting real world attacks.
Can we learn where people come from? Retracing of origins in merging situations
Venue: arXiv cs.AI 2020
Year: 2020

One crucial information for a pedestrian crowd simulation is the number of agents moving from an origin to a certain target. While this setup has a large impact on the simulation, it is in most setups challenging to find the number of agents that should be spawned at a source in the simulation. Often, number are chosen based on surveys and experience of modelers and event organizers. These approaches are important and useful but reach their limits when we want to perform real-time predictions. In this case, a static information about the inflow is not sufficient. Instead, we need a dynamic information that can be retrieved each time the prediction is started. Nowadays, sensor data such as video footage or GPS tracks of a crowd are often available. If we can estimate the number of pedestrians who stem from a certain origin from this sensor data, we can dynamically initialize the simulation. In this study, we use density heatmaps that can be derived from sensor data as input for a random forest regressor to predict the origin distributions. We study three different datasets: A simulated dataset, experimental data, and a hybrid approach with both experimental and simulated data. In the hybrid setup, the model is trained with simulated data and then tested on experimental data. The results demonstrate that the random forest model is able to predict the origin distribution based on a single density heatmap for all three configurations. This is especially promising for applying the approach on real data since there is often only a limited amount of data available.
Reduced Graphene Oxide Tattoo as Wearable Proximity Sensor
Venue: arXiv cs.HC 2020
Year: 2020

The human body is punctuated with wide array of sensory systems that provide a high evolutionary advantage by facilitating formation of a detailed picture of the immediate surroundings. The sensors range across a wide spectrum, acquiring input from non-contact audio-visual means to contact based input via pressure and temperature. The ambit of sensing can be extended further by imparting the body with increased non-contact sensing capability through the phenomenon of electrostatics. Here we present graphene-based tattoo sensor for proximity sensing, employing the principle of electrostatic gating. The sensor shows a remarkable change in resistance upon exposure to objects surrounded with static charge on them. Compared to prior work in this field, the sensor has demonstrated the highest recorded proximity detection range of 20 cm. It is ultra-thin, highly skin conformal and comes with a facile transfer process such that it can be tattooed on highly curvilinear rough substrates like the human skin, unlike other graphene-based proximity sensors reported before. Present work details the operation of wearable proximity sensor while exploring the effect of mounting body on the working mechanism. A possible role of the sensor as an alerting system against unwarranted contact with objects in public places especially during the current SARS-CoV-2 pandemic has also been explored in the form of an LED bracelet whose color is controlled by the proximity sensor attached to it.
Privacy Interpretation of Behavioural-based Anomaly Detection Approaches
Venue: arXiv cs.CR 2020
Year: 2020

This paper proposes the notion of 'Privacy-Anomaly Detection' and considers the question of whether behavioural-based anomaly detection approaches can have a privacy semantic interpretation and whether the detected anomalies can be related to the conventional (formal) definitions of privacy semantics such as k-anonymity. The idea is to learn the user's past querying behaviour in terms of privacy and then identifying deviations from past behaviour in order to detect privacy violations. Privacy attacks, violations of formal privacy definition, based on a sequence of SQL queries (query correlations) are also considered in the paper and it is shown that interactive querying settings are vulnerable to privacy attacks based on query sequences. Investigation on whether these types of privacy attacks can potentially manifest themselves as anomalies, specifically as privacy-anomalies was carried out. It is shown that in this paper that behavioural-based anomaly detection approaches have the potential to detect privacy attacks based on query sequences (violation of formal privacy definition) as privacy-anomalies.
BERTChem-DDI : Improved Drug-Drug Interaction Prediction from text using Chemical Structure Information
Venue: arXiv cs.CL 2020
Year: 2020

Traditional biomedical version of embeddings obtained from pre-trained language models have recently shown state-of-the-art results for relation extraction (RE) tasks in the medical domain. In this paper, we explore how to incorporate domain knowledge, available in the form of molecular structure of drugs, for predicting Drug-Drug Interaction from textual corpus. We propose a method, BERTChem-DDI, to efficiently combine drug embeddings obtained from the rich chemical structure of drugs along with off-the-shelf domain-specific BioBERT embedding-based RE architecture. Experiments conducted on the DDIExtraction 2013 corpus clearly indicate that this strategy improves other strong baselines architectures by 3.4\% macro F1-score.
Exploring and Analyzing Machine Commonsense Benchmarks
Venue: arXiv cs.AI 2020
Year: 2020

Commonsense question-answering (QA) tasks, in the form of benchmarks, are constantly being introduced for challenging and comparing commonsense QA systems. The benchmarks provide question sets that systems' developers can use to train and test new models before submitting their implementations to official leaderboards. Although these tasks are created to evaluate systems in identified dimensions (e.g. topic, reasoning type), this metadata is limited and largely presented in an unstructured format or completely not present. Because machine common sense is a fast-paced field, the problem of fully assessing current benchmarks and systems with regards to these evaluation dimensions is aggravated. We argue that the lack of a common vocabulary for aligning these approaches' metadata limits researchers in their efforts to understand systems' deficiencies and in making effective choices for future tasks. In this paper, we first discuss this MCS ecosystem in terms of its elements and their metadata. Then, we present how we are supporting the assessment of approaches by initially focusing on commonsense benchmarks. We describe our initial MCS Benchmark Ontology, an extensible common vocabulary that formalizes benchmark metadata, and showcase how it is supporting the development of a Benchmark tool that enables benchmark exploration and analysis.
myGym: Modular Toolkit for Visuomotor Robotic Tasks
Venue: arXiv cs.RO 2020
Year: 2020

We introduce a novel virtual robotic toolkit myGym, developed for reinforcement learning (RL), intrinsic motivation and imitation learning tasks trained in a 3D simulator. The trained tasks can then be easily transferred to real-world robotic scenarios. The modular structure of the simulator enables users to train and validate their algorithms on a large number of scenarios with various robots, environments and tasks. Compared to existing toolkits (e.g. OpenAI Gym, Roboschool) which are suitable for classical RL, myGym is also prepared for visuomotor (combining vision & movement) unsupervised tasks that require intrinsic motivation, i.e. the robots are able to generate their own goals. There are also collaborative scenarios intended for human-robot interaction. The toolkit provides pretrained visual modules for visuomotor tasks allowing rapid prototyping, and, moreover, users can customize the visual submodules and retrain with their own set of objects. In practice, the user selects the desired environment, robot, objects, task and type of reward as simulation parameters, and the training, visualization and testing themselves are handled automatically. The user can thus fully focus on development of the neural network architecture while controlling the behaviour of the environment using predefined parameters.
Scheduling Coflows with Dependency Graph
Venue: arXiv cs.DC 2020
Year: 2020

Applications in data-parallel computing typically consist of multiple stages. In each stage, a set of intermediate parallel data flows (Coflow) is produced and transferred between servers to enable starting of next stage. While there has been much research on scheduling isolated coflows, the dependency between coflows in multi-stage jobs has been largely ignored. In this paper, we consider scheduling coflows of multi-stage jobs represented by general DAGs (Directed Acyclic Graphs) in a shared data center network, so as to minimize the total weighted completion time of jobs. This problem is significantly more challenging than the traditional coflow scheduling, as scheduling even a single multi-stage job to minimize its completion time is shown to be NP-hard. In this paper, we propose a polynomial-time algorithm with approximation ratio of $O(μ\log(m)/\log(\log(m)))$, where $μ$ is the maximum number of coflows in a job and $m$ is the number of servers. For the special case that the jobs' underlying dependency graphs are rooted trees, we modify the algorithm and improve its approximation ratio. To verify the performance of our algorithms, we present simulation results using real traffic traces that show up to $53 \%$ improvement over the prior approach. We conclude the paper by providing a result concerning an optimality gap for scheduling coflows with general DAGs.
An Architecture for In-Vehicle Network
Venue: arXiv cs.NI 2020
Year: 2020

As vehicles get equipped with increasingly complex sensors and processors, the communication requirements become more demanding. Traditionally, vehicles have used specialized networking technologies designed to guarantee bounded latencies, such as the Controller Area Network (CAN) bus. Recently, some have used dedicated technologies to transport signals from cameras, lidars, radars, and ultrasonic sensors. In parallel, IEEE working groups are defining Ethernet standards for time-sensitive networks (TSN). This paper describes an Ethernet-based architecture with provable guaranteed performance and simple configuration that is suitable for supporting the communication requirements of many vehicles.
Cross-Domain Latent Modulation for Variational Transfer Learning
Venue: arXiv cs.LG 2020
Year: 2020

We propose a cross-domain latent modulation mechanism within a variational autoencoders (VAE) framework to enable improved transfer learning. Our key idea is to procure deep representations from one data domain and use it as perturbation to the reparameterization of the latent variable in another domain. Specifically, deep representations of the source and target domains are first extracted by a unified inference model and aligned by employing gradient reversal. Second, the learned deep representations are cross-modulated to the latent encoding of the alternate domain. The consistency between the reconstruction from the modulated latent encoding and the generation using deep representation samples is then enforced in order to produce inter-class alignment in the latent space. We apply the proposed model to a number of transfer learning tasks including unsupervised domain adaptation and image-toimage translation. Experimental results show that our model gives competitive performance.
Predicting the Critical Number of Layers for Hierarchical Support Vector Regression
Venue: arXiv cs.LG 2020
Year: 2020

Hierarchical support vector regression (HSVR) models a function from data as a linear combination of SVR models at a range of scales, starting at a coarse scale and moving to finer scales as the hierarchy continues. In the original formulation of HSVR, there were no rules for choosing the depth of the model. In this paper, we observe in a number of models a phase transition in the training error -- the error remains relatively constant as layers are added, until a critical scale is passed, at which point the training error drops close to zero and remains nearly constant for added layers. We introduce a method to predict this critical scale a priori with the prediction based on the support of either a Fourier transform of the data or the Dynamic Mode Decomposition (DMD) spectrum. This allows us to determine the required number of layers prior to training any models.
GuidedStyle: Attribute Knowledge Guided Style Manipulation for Semantic Face Editing
Venue: arXiv cs.CV 2020
Year: 2020

Although significant progress has been made in synthesizing high-quality and visually realistic face images by unconditional Generative Adversarial Networks (GANs), there still lacks of control over the generation process in order to achieve semantic face editing. In addition, it remains very challenging to maintain other face information untouched while editing the target attributes. In this paper, we propose a novel learning framework, called GuidedStyle, to achieve semantic face editing on StyleGAN by guiding the image generation process with a knowledge network. Furthermore, we allow an attention mechanism in StyleGAN generator to adaptively select a single layer for style manipulation. As a result, our method is able to perform disentangled and controllable edits along various attributes, including smiling, eyeglasses, gender, mustache and hair color. Both qualitative and quantitative results demonstrate the superiority of our method over other competing methods for semantic face editing. Moreover, we show that our model can be also applied to different types of real and artistic face editing, demonstrating strong generalization ability.