OpenRaiser/Intern-Atlas huggingface dataset:
Causal Mechanism Transfer Network for Time Series Domain Adaptation in Mechanical Systems
Venue: arXiv cs.LG 2019
Year: 2019

Data-driven models are becoming essential parts in modern mechanical systems, commonly used to capture the behavior of various equipment and varying environmental characteristics. Despite the advantages of these data-driven models on excellent adaptivity to high dynamics and aging equipment, they are usually hungry to massive labels over historical data, mostly contributed by human engineers at an extremely high cost. The label demand is now the major limiting factor to modeling accuracy, hindering the fulfillment of visions for applications. Fortunately, domain adaptation enhances the model generalization by utilizing the labelled source data as well as the unlabelled target data and then we can reuse the model on different domains. However, the mainstream domain adaptation methods cannot achieve ideal performance on time series data, because most of them focus on static samples and even the existing time series domain adaptation methods ignore the properties of time series data, such as temporal causal mechanism. In this paper, we assume that causal mechanism is invariant and present our Causal Mechanism Transfer Network(CMTN) for time series domain adaptation. By capturing and transferring the dynamic and temporal causal mechanism of multivariate time series data and alleviating the time lags and different value ranges among different machines, CMTN allows the data-driven models to exploit existing data and labels from similar systems, such that the resulting model on a new system is highly reliable even with very limited data. We report our empirical results and lessons learned from two real-world case studies, on chiller plant energy optimization and boiler fault detection, which outperforms the existing state-of-the-art method.
Trajectory Advancement for Robot Stand-up with Human Assistance
Venue: arXiv cs.RO 2019
Year: 2019

Physical interactions are inevitable part of human-robot collaboration tasks and rather than exhibiting simple reactive behaviors to human interactions, collaborative robots need to be endowed with intuitive behaviors. This paper proposes a trajectory advancement approach that facilitates advancement along a reference trajectory by leveraging assistance from helpful interaction wrench present during human-robot collaboration. We validate our approach through experiments in simulation with iCub.
REVE: Regularizing Deep Learning with Variational Entropy Bound
Venue: arXiv cs.LG 2019
Year: 2019

Studies on generalization performance of machine learning algorithms under the scope of information theory suggest that compressed representations can guarantee good generalization, inspiring many compression-based regularization methods. In this paper, we introduce REVE, a new regularization scheme. Noting that compressing the representation can be sub-optimal, our first contribution is to identify a variable that is directly responsible for the final prediction. Our method aims at compressing the class conditioned entropy of this latter variable. Second, we introduce a variational upper bound on this conditional entropy term. Finally, we propose a scheme to instantiate a tractable loss that is integrated within the training procedure of the neural network and demonstrate its efficiency on different neural networks and datasets.
Statically Detecting Vulnerabilities by Processing Programming Languages as Natural Languages
Venue: arXiv cs.CR 2019
Year: 2019

Web applications continue to be a favorite target for hackers due to a combination of wide adoption and rapid deployment cycles, which often lead to the introduction of high impact vulnerabilities. Static analysis tools are important to search for bugs automatically in the program source code, supporting developers on their removal. However, building these tools requires programming the knowledge on how to discover the vulnerabilities. This paper presents an alternative approach in which tools learn to detect flaws automatically by resorting to artificial intelligence concepts, more concretely to natural language processing. The approach employs a sequence model to learn to characterize vulnerabilities based on an annotated corpus. Afterwards, the model is utilized to discover and identify vulnerabilities in the source code. It was implemented in the DEKANT tool and evaluated experimentally with a large set of PHP applications and WordPress plugins. Overall, we found several hundred vulnerabilities belonging to 12 classes of input validation vulnerabilities, where 62 of them were zero-day.
Reversible cyclic codes over $\mathbb{F}_q + u \mathbb{F}_q$
Venue: arXiv cs.IT 2019
Year: 2019

Let $q$ be a power of a prime $p$. In this paper, we study reversible cyclic codes of arbitrary length over the ring $ R = \mathbb{F}_q + u \mathbb{F}_q$, where $u^2=0 mod q$. First, we find a unique set of generators for cyclic codes over $R$, followed by a classification of reversible cyclic codes with respect to their generators. Also, under certain conditions, it is shown that dual of reversible cyclic code is reversible over $\mathbb{Z}_2+u\mathbb{Z}_2$. Further, to show the importance of these results, some examples of reversible cyclic codes are provided.
Breadth-first, Depth-next Training of Random Forests
Venue: arXiv cs.LG 2019
Year: 2019

In this paper we analyze, evaluate, and improve the performance of training Random Forest (RF) models on modern CPU architectures. An exact, state-of-the-art binary decision tree building algorithm is used as the basis of this study. Firstly, we investigate the trade-offs between using different tree building algorithms, namely breadth-first-search (BFS) and depth-search-first (DFS). We design a novel, dynamic, hybrid BFS-DFS algorithm and demonstrate that it performs better than both BFS and DFS, and is more robust in the presence of workloads with different characteristics. Secondly, we identify CPU performance bottlenecks when generating trees using this approach, and propose optimizations to alleviate them. The proposed hybrid tree building algorithm for RF is implemented in the Snap Machine Learning framework, and speeds up the training of RFs by 7.8x on average when compared to state-of-the-art RF solvers (sklearn, H2O, and xgboost) on a range of datasets, RF configurations, and multi-core CPU architectures.
Quantifying Classification Uncertainty using Regularized Evidential Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Traditional deep neural nets (NNs) have shown the state-of-the-art performance in the task of classification in various applications. However, NNs have not considered any types of uncertainty associated with the class probabilities to minimize risk due to misclassification under uncertainty in real life. Unlike Bayesian neural nets indirectly infering uncertainty through weight uncertainties, evidential neural networks (ENNs) have been recently proposed to support explicit modeling of the uncertainty of class probabilities. It treats predictions of an NN as subjective opinions and learns the function by collecting the evidence leading to these opinions by a deterministic NN from data. However, an ENN is trained as a black box without explicitly considering different types of inherent data uncertainty, such as vacuity (uncertainty due to a lack of evidence) or dissonance (uncertainty due to conflicting evidence). This paper presents a new approach, called a {\em regularized ENN}, that learns an ENN based on regularizations related to different characteristics of inherent data uncertainty. Via the experiments with both synthetic and real-world datasets, we demonstrate that the proposed regularized ENN can better learn of an ENN modeling different types of uncertainty in the class probabilities for classification tasks.
Blockchain of Things (BCoT): The Fusion of Blockchain and IoT Technologies
Venue: arXiv cs.NI 2019
Year: 2019

Blockchain, as well as Internet of Things (IoT), is considered as two major disruptive emerging technologies. However, both of them suffer from innate technological limitations to some extent. IoT requires strengthening its security features while Blockchain inherently possesses them due to its extensive use of cryptographic mechanisms and Blockchain, in an inverted manner, needs contributions from the distributed nodes for its P2P (Peer-to-Peer) consensus model while IoT rudimentarily embodies them within its architecture. This chapter, therefore, acutely dissects the viability, along with prospective challenges, of incorporating Blockchain with IoT technologies,inducing the notion of Blockchain of Things (BCoT), as well as the benefits such consolidation can offer.
Human Action Recognition with Multi-Laplacian Graph Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

Convolutional neural networks are nowadays witnessing a major success in different pattern recognition problems. These learning models were basically designed to handle vectorial data such as images but their extension to non-vectorial and semi-structured data (namely graphs with variable sizes, topology, etc.) remains a major challenge, though a few interesting solutions are currently emerging. In this paper, we introduce MLGCN; a novel spectral Multi-Laplacian Graph Convolutional Network. The main contribution of this method resides in a new design principle that learns graph-laplacians as convex combinations of other elementary laplacians each one dedicated to a particular topology of the input graphs. We also introduce a novel pooling operator, on graphs, that proceeds in two steps: context-dependent node expansion is achieved, followed by a global average pooling; the strength of this two-step process resides in its ability to preserve the discrimination power of nodes while achieving permutation invariance. Experiments conducted on SBU and UCF-101 datasets, show the validity of our method for the challenging task of action recognition.
Orthogonal Gradient Descent for Continual Learning
Venue: arXiv cs.LG 2019
Year: 2019

Neural networks are achieving state of the art and sometimes super-human performance on learning tasks across a variety of domains. Whenever these problems require learning in a continual or sequential manner, however, neural networks suffer from the problem of catastrophic forgetting; they forget how to solve previous tasks after being trained on a new task, despite having the essential capacity to solve both tasks if they were trained on both simultaneously. In this paper, we propose to address this issue from a parameter space perspective and study an approach to restrict the direction of the gradient updates to avoid forgetting previously-learned data. We present the Orthogonal Gradient Descent (OGD) method, which accomplishes this goal by projecting the gradients from new tasks onto a subspace in which the neural network output on previous task does not change and the projected gradient is still in a useful direction for learning the new task. Our approach utilizes the high capacity of a neural network more efficiently and does not require storing the previously learned data that might raise privacy concerns. Experiments on common benchmarks reveal the effectiveness of the proposed OGD method.
Hyper: Distributed Cloud Processing for Large-Scale Deep Learning Tasks
Venue: arXiv cs.DC 2019
Year: 2019

Training and deploying deep learning models in real-world applications require processing large amounts of data. This is a challenging task when the amount of data grows to a hundred terabytes, or even, petabyte-scale. We introduce a hybrid distributed cloud framework with a unified view to multiple clouds and an on-premise infrastructure for processing tasks using both CPU and GPU compute instances at scale. The system implements a distributed file system and failure-tolerant task processing scheduler, independent of the language and Deep Learning framework used. It allows to utilize unstable cheap resources on the cloud to significantly reduce costs. We demonstrate the scalability of the framework on running pre-processing, distributed training, hyperparameter search and large-scale inference tasks utilizing 10,000 CPU cores and 300 GPU instances with the overall processing power of 30 petaflops.
Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation
Venue: arXiv cs.LG 2019
Year: 2019

Infinite horizon off-policy policy evaluation is a highly challenging task due to the excessively large variance of typical importance sampling (IS) estimators. Recently, Liu et al. (2018a) proposed an approach that significantly reduces the variance of infinite-horizon off-policy evaluation by estimating the stationary density ratio, but at the cost of introducing potentially high biases due to the error in density ratio estimation. In this paper, we develop a bias-reduced augmentation of their method, which can take advantage of a learned value function to obtain higher accuracy. Our method is doubly robust in that the bias vanishes when either the density ratio or the value function estimation is perfect. In general, when either of them is accurate, the bias can also be reduced. Both theoretical and empirical results show that our method yields significant advantages over previous methods.
Tutorial on NLP-Inspired Network Embedding
Venue: arXiv cs.SI 2019
Year: 2019

This tutorial covers a few recent papers in the field of network embedding. Network embedding is a collective term for techniques for mapping graph nodes to vectors of real numbers in a multidimensional space. To be useful, a good embedding should preserve the structure of the graph. The vectors can then be used as input to various network and graph analysis tasks, such as link prediction. The papers discussed develop methods for the online learning of such embeddings, and include DeepWalk, LINE, node2vec, struc2vec and megapath2vec. These new methods and developments in online learning of network embeddings have major applications for the analysis of graphs and networks, including online social networks.
DevOps in Practice -- A preliminary Analysis of two Multinational Companies
Venue: arXiv cs.SE 2019
Year: 2019

DevOps is a cultural movement that aims the collaboration of all the stakeholders involved in the development, deployment and operation of soft-ware to deliver a quality product or service in the shortest possible time. DevOps is relatively recent, and companies have developed their DevOps prac-tices largely from scratch. Our research aims to conduct an analysis on practic-ing DevOps in +20 software-intensive companies to provide patterns of DevOps practices and identify their benefits and barriers. This paper presents the preliminary analysis of an exploratory case study based on the interviews to relevant stakeholders of two (multinational) companies. The results show the benefits (software delivery performance) and barriers that these companies are dealing with, as well as DevOps team topology they approached during their DevOps transformation. This study aims to help practitioners and researchers to better understand DevOps transformations and the contexts where the practices worked. This, hopefully, will contribute to strengthening the evidence regarding DevOps and supporting practitioners in making better informed decisions about the return of investment when adopting DevOps.
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
Venue: arXiv cs.LG 2019
Year: 2019

We consider the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments. To do so, we study how a teacher algorithm can learn to generate a learning curriculum, whereby it sequentially samples parameters controlling a stochastic procedural generation of environments. Because it does not initially know the capacities of its student, a key challenge for the teacher is to discover which environments are easy, difficult or unlearnable, and in what order to propose them to maximize the efficiency of learning over the learnable ones. To achieve this, this problem is transformed into a surrogate continuous bandit problem where the teacher samples environments in order to maximize absolute learning progress of its student. We present a new algorithm modeling absolute learning progress with Gaussian mixture models (ALP-GMM). We also adapt existing algorithms and provide a complete study in the context of DRL. Using parameterized variants of the BipedalWalker environment, we study their efficiency to personalize a learning curriculum for different learners (embodiments), their robustness to the ratio of learnable/unlearnable environments, and their scalability to non-linear and high-dimensional parameter spaces. Videos and code are available at https://github.com/flowersteam/teachDeepRL.
Long Optimal or Small-Defect LRC Codes with Unbounded Minimum Distances
Venue: arXiv cs.IT 2019
Year: 2019

A code over a finite field is called locally recoverable code (LRC) if every coordinate symbol can be determined by a small number (at most r, this parameter is called locality) of other coordinate symbols. For a linear code with length n, dimension k and locality r, its minimum distance d satisfies a Singleton-like bound. A code attaining this bound is called optimal. Many families of optimal locally recoverable codes have been constructed by using different techniques in finite fields or algebraic curves. However no optimal LRC code over a general finite field of q elements with the length n around the square of q, the locality r larger than or equal to 24 and the minimum distance d larger than or equal to 9 has been constructed. In this paper for any given finite field of q elements, any given r between 1 and q-1 and given d in certain range, we give an optimal LRC code with length n around the square of q, locality r and minimum distance d. This is the only known family of optimal LRC codes with lengths n around the square of q and unbounded localities and minimum distances d larger than or equal to 9. We also gives an asymptotic bound for q-ary r locality LRC codes better than the previously konwn bound. Many long r-locality LRC codes with small defects are also constructed.
Universal Bounds for Size and Energy of Codes of Given Minimum and Maximum Distances
Venue: arXiv cs.IT 2019
Year: 2019

We employ signed measures that are positive definite up to certain degrees to establish Levenshtein-type upper bounds on the cardinality of codes with given minimum and maximum distances, and universal lower bounds on the potential energy (for absolutely monotone interactions) for codes with given maximum distance and cardinality. The distance distributions of codes that attain the bounds are found in terms of the parameters of Levenshtein-type quadrature formulas. Necessary and sufficient conditions for the optimality of our bounds are derived. Further, we obtain upper bounds on the energy of codes of fixed minimum and maximum distances and cardinality.
An Adaptive Broadcasting Strategy for Efficient Dynamic Mapping in Vehicular Networks
Venue: arXiv cs.NI 2019
Year: 2019

In this work, we face the issue of achieving an efficient dynamic mapping in vehicular networking scenarios, i.e., to obtain an accurate estimate of the positions and trajectories of connected vehicles in a certain area. State of the art solutions are based on the periodic broadcasting of the position information of the network nodes, with an inter-transmission period set by a congestion control scheme. However, the movements and maneuvers of vehicles can often be erratic, making transmitted data inaccurate or downright misleading. To address this problem, we propose to adopt a dynamic transmission scheme based on the actual positioning error, sending new data when the estimate passes a preset error threshold. Furthermore, the proposed method adapts the error threshold to the operational context according to a congestion control algorithm that limits the collision probability among broadcast packet transmissions. This threshold-based strategy can reduce the network load by avoiding the transmission of redundant messages, and is shown to improve the overall positioning accuracy by more than 20% in realistic urban scenarios.
Hiding data inside images using orthogonal moments
Venue: arXiv cs.MM 2019
Year: 2019

In this contribution we propose a novel steganographic method based on several orthogonal polynomials and their combinations. The steganographic algorithm embeds a secrete message at the first eight coefficients of high frequency image. Moreover, this embedding method uses the Beta chaotic map to determine the order of the blocks where the secret bits will be inserted. In addition, from a 128-bit private key and the steps of a cryptography algorithm according to the Advanced Encryption Standard (AES) to generate the key expansion, the proposed method generates a key expansion of 2560 bits, with the purpose to permute the first eight coefficients of high frequency before the insertion. The insertion takes eventually place at the first eight high frequency coefficients in the transformed orthogonal moments domain. Before the insertion of the message the image undergoes a series of transformations. After the insertion the inverse transformations are applied to the original transformations in reverse order. The experimental work on the validation of the algorithm consists of the calculation of the Peak Signal-to-Noise Ratio (PSNR), the Universal Image Quality Index (UIQI), the Image Fidelity (IF), and the Relative Entropy (RE), comparing the same characteristics for the cover and stego image. The proposed algorithm improves the level of imperceptibility and security analyzed through the PSNR and RE values, respectively.
Do Explanations Reflect Decisions? A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms
Venue: arXiv cs.LG 2019
Year: 2019

There has been a significant surge of interest recently around the concept of explainable artificial intelligence (XAI), where the goal is to produce an interpretation for a decision made by a machine learning algorithm. Of particular interest is the interpretation of how deep neural networks make decisions, given the complexity and `black box' nature of such networks. Given the infancy of the field, there has been very limited exploration into the assessment of the performance of explainability methods, with most evaluations centered around subjective visual interpretation of the produced interpretations. In this study, we explore a more machine-centric strategy for quantifying the performance of explainability methods on deep neural networks via the notion of decision-making impact analysis. We introduce two quantitative performance metrics: i) Impact Score, which assesses the percentage of critical factors with either strong confidence reduction impact or decision changing impact, and ii) Impact Coverage, which assesses the percentage coverage of adversarially impacted factors in the input. A comprehensive analysis using this approach was conducted on several state-of-the-art explainability methods (LIME, SHAP, Expected Gradients, GSInquire) on a ResNet-50 deep convolutional neural network using a subset of ImageNet for the task of image classification. Experimental results show that the critical regions identified by LIME within the tested images had the lowest impact on the decision-making process of the network (~38%), with progressive increase in decision-making impact for SHAP (~44%), Expected Gradients (~51%), and GSInquire (~76%). While by no means perfect, the hope is that the proposed machine-centric strategy helps push the conversation forward towards better metrics for evaluating explainability methods and improve trust in deep neural networks.