OpenRaiser/Intern-Atlas huggingface dataset:
Truncated Back-propagation for Bilevel Optimization
Venue: arXiv cs.LG 2019
Year: 2019

Bilevel optimization has been recently revisited for designing and analyzing algorithms in hyperparameter tuning and meta learning tasks. However, due to its nested structure, evaluating exact gradients for high-dimensional problems is computationally challenging. One heuristic to circumvent this difficulty is to use the approximate gradient given by performing truncated back-propagation through the iterative optimization procedure that solves the lower-level problem. Although promising empirical performance has been reported, its theoretical properties are still unclear. In this paper, we analyze the properties of this family of approximate gradients and establish sufficient conditions for convergence. We validate this on several hyperparameter tuning and meta learning tasks. We find that optimization with the approximate gradient computed using few-step back-propagation often performs comparably to optimization with the exact gradient, while requiring far less memory and half the computation time.
An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection
Venue: arXiv cs.CV 2019
Year: 2019

The identification of lesion within medical image data is necessary for diagnosis, treatment and prognosis. Segmentation and classification approaches are mainly based on supervised learning with well-paired image-level or voxel-level labels. However, labeling the lesion in medical images is laborious requiring highly specialized knowledge. We propose a medical image synthesis model named abnormal-to-normal translation generative adversarial network (ANT-GAN) to generate a normal-looking medical image based on its abnormal-looking counterpart without the need for paired training data. Unlike typical GANs, whose aim is to generate realistic samples with variations, our more restrictive model aims at producing a normal-looking image corresponding to one containing lesions, and thus requires a special design. Being able to provide a "normal" counterpart to a medical image can provide useful side information for medical imaging tasks like lesion segmentation or classification validated by our experiments. In the other aspect, the ANT-GAN model is also capable of producing highly realistic lesion-containing image corresponding to the healthy one, which shows the potential in data augmentation verified in our experiments.
Evading classifiers in discrete domains with provable optimality guarantees
Venue: arXiv cs.LG 2019
Year: 2019

Machine-learning models for security-critical applications such as bot, malware, or spam detection, operate in constrained discrete domains. These applications would benefit from having provable guarantees against adversarial examples. The existing literature on provable adversarial robustness of models, however, exclusively focuses on robustness to gradient-based attacks in domains such as images. These attacks model the adversarial cost, e.g., amount of distortion applied to an image, as a $p$-norm. We argue that this approach is not well-suited to model adversarial costs in constrained domains where not all examples are feasible. We introduce a graphical framework that (1) generalizes existing attacks in discrete domains, (2) can accommodate complex cost functions beyond $p$-norms, including financial cost incurred when attacking a classifier, and (3) efficiently produces valid adversarial examples with guarantees of minimal adversarial cost. These guarantees directly translate into a notion of adversarial robustness that takes into account domain constraints and the adversary's capabilities. We show how our framework can be used to evaluate security by crafting adversarial examples that evade a Twitter-bot detection classifier with provably minimal number of changes; and to build privacy defenses by crafting adversarial examples that evade a privacy-invasive website-fingerprinting classifier.
Development and Analysis of Deterministic Privacy-Preserving Policies Using Non-Stochastic Information Theory
Venue: arXiv cs.IT 2019
Year: 2019

A deterministic privacy metric using non-stochastic information theory is developed. Particularly, minimax information is used to construct a measure of information leakage, which is inversely proportional to the measure of privacy. Anyone can submit a query to a trusted agent with access to a non-stochastic uncertain private dataset. Optimal deterministic privacy-preserving policies for responding to the submitted query are computed by maximizing the measure of privacy subject to a constraint on the worst-case quality of the response (i.e., the worst-case difference between the response by the agent and the output of the query computed on the private dataset). The optimal privacy-preserving policy is proved to be a piecewise constant function in the form of a quantization operator applied on the output of the submitted query. The measure of privacy is also used to analyze the performance of $k$-anonymity methodology (a popular deterministic mechanism for privacy-preserving release of datasets using suppression and generalization techniques), proving that it is in fact not privacy-preserving.
CrystalGAN: Learning to Discover Crystallographic Structures with Generative Adversarial Networks
Venue: arXiv cs.LG 2019
Year: 2019

Our main motivation is to propose an efficient approach to generate novel multi-element stable chemical compounds that can be used in real world applications. This task can be formulated as a combinatorial problem, and it takes many hours of human experts to construct, and to evaluate new data. Unsupervised learning methods such as Generative Adversarial Networks (GANs) can be efficiently used to produce new data. Cross-domain Generative Adversarial Networks were reported to achieve exciting results in image processing applications. However, in the domain of materials science, there is a need to synthesize data with higher order complexity compared to observed samples, and the state-of-the-art cross-domain GANs can not be adapted directly. In this contribution, we propose a novel GAN called CrystalGAN which generates new chemically stable crystallographic structures with increased domain complexity. We introduce an original architecture, we provide the corresponding loss functions, and we show that the CrystalGAN generates very reasonable data. We illustrate the efficiency of the proposed method on a real original problem of novel hydrides discovery that can be further used in development of hydrogen storage materials.
Online Fault Classification in HPC Systems through Machine Learning
Venue: arXiv cs.DC 2019
Year: 2019

As High-Performance Computing (HPC) systems strive towards the exascale goal, studies suggest that they will experience excessive failure rates. For this reason, detecting and classifying faults in HPC systems as they occur and initiating corrective actions before they can transform into failures will be essential for continued operation. In this paper, we propose a fault classification method for HPC systems based on machine learning that has been designed specifically to operate with live streamed data. We cast the problem and its solution within realistic operating constraints of online use. Our results show that almost perfect classification accuracy can be reached for different fault types with low computational overhead and minimal delay. We have based our study on a local dataset, which we make publicly available, that was acquired by injecting faults to an in-house experimental HPC system.
Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a new approach for facial expression recognition using deep covariance descriptors. The solution is based on the idea of encoding local and global Deep Convolutional Neural Network (DCNN) features extracted from still images, in compact local and global covariance descriptors. The space geometry of the covariance matrices is that of Symmetric Positive Definite (SPD) matrices. By conducting the classification of static facial expressions using Support Vector Machine (SVM) with a valid Gaussian kernel on the SPD manifold, we show that deep covariance descriptors are more effective than the standard classification with fully connected layers and softmax. Besides, we propose a completely new and original solution to model the temporal dynamic of facial expressions as deep trajectories on the SPD manifold. As an extension of the classification pipeline of covariance descriptors, we apply SVM with valid positive definite kernels derived from global alignment for deep covariance trajectories classification. By performing extensive experiments on the Oulu-CASIA, CK+, and SFEW datasets, we show that both the proposed static and dynamic approaches achieve state-of-the-art performance for facial expression recognition outperforming many recent approaches.
Groupcast Index Coding Problem: Joint Extensions
Venue: arXiv cs.IT 2019
Year: 2019

The groupcast index coding problem is the most general version of the classical index coding problem, where any receiver can demand messages that are also demanded by other receivers. Any groupcast index coding problem is described by its \emph{fitting matrix} which contains unknown entries along with $1$'s and $0$'s. The problem of finding an optimal scalar linear code is equivalent to completing this matrix with known entries such that the rank of the resulting matrix is minimized. Any row basis of such a completion gives an optimal \emph{scalar linear} code. An index coding problem is said to be a joint extension of a finite number of index coding problems, if the fitting matrices of these problems are disjoint submatrices of the fitting matrix of the jointly extended problem. In this paper, a class of joint extensions of any finite number of groupcast index coding problems is identified, where the relation between the fitting matrices of the sub-problems present in the fitting matrix of the jointly extended problem is defined by a base problem. A lower bound on the \emph{minrank} (optimal scalar linear codelength) of the jointly extended problem is given in terms of those of the sub-problems. This lower bound also has a dependence on the base problem and is operationally useful in finding lower bounds of the jointly extended problems when the minranks of all the sub-problems are known. We provide an algorithm to construct scalar linear codes (not optimal in general), for any groupcast problem belonging to the class of jointly extended problems identified in this paper. The algorithm uses scalar linear codes of all the sub-problems and the base problem. We also identify some subclasses, where the constructed codes are scalar linear optimal.
Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis
Venue: arXiv cs.CV 2019
Year: 2019

Despite remarkable advances in image synthesis research, existing works often fail in manipulating images under the context of large geometric transformations. Synthesizing person images conditioned on arbitrary poses is one of the most representative examples where the generation quality largely relies on the capability of identifying and modeling arbitrary transformations on different body parts. Current generative models are often built on local convolutions and overlook the key challenges (e.g. heavy occlusions, different views or dramatic appearance changes) when distinct geometric changes happen for each part, caused by arbitrary pose manipulations. This paper aims to resolve these challenges induced by geometric variability and spatial displacements via a new Soft-Gated Warping Generative Adversarial Network (Warping-GAN), which is composed of two stages: 1) it first synthesizes a target part segmentation map given a target pose, which depicts the region-level spatial layouts for guiding image synthesis with higher-level structure constraints; 2) the Warping-GAN equipped with a soft-gated warping-block learns feature-level mapping to render textures from the original image into the generated segmentation map. Warping-GAN is capable of controlling different transformation degrees given distinct target poses. Moreover, the proposed warping-block is light-weight and flexible enough to be injected into any networks. Human perceptual studies and quantitative evaluations demonstrate the superiority of our Warping-GAN that significantly outperforms all existing methods on two large datasets.
Minimum Reload Cost Graph Factors
Venue: arXiv cs.CC 2019
Year: 2019

The concept of Reload cost in a graph refers to the cost that occurs while traversing a vertex via two of its incident edges. This cost is uniquely determined by the colors of the two edges. This concept has various applications in transportation networks, communication networks, and energy distribution networks. Various problems using this model are defined and studied in the literature. The problem of finding a spanning tree whose diameter with respect to the reload costs is the smallest possible, the problems of finding a path, trail or walk with minimum total reload cost between two given vertices, problems about finding a proper edge coloring of a graph such that the total reload cost is minimized, the problem of finding a spanning tree such that the sum of the reload costs of all paths between all pairs of vertices is minimized, and the problem of finding a set of cycles of minimum reload cost, that cover all the vertices of a graph, are examples of such problems. % In this work we focus on the last problem. Noting that a cycle cover of a graph is a 2-factor of it, we generalize the problem to that of finding an $r$-factor of minimum reload cost of an edge colored graph. We prove several NP-hardness results for special cases of the problem. Namely, bounded degree graphs, planar graphs, bounded total cost, and bounded number of distinct costs. For the special case of $r=2$, our results imply an improved NP-hardness result. On the positive side, we present a polynomial-time solvable special case which provides a tight boundary between the polynomial and hard cases in terms of $r$ and the maximum degree of the graph. We then investigate the parameterized complexity of the problem, prove W[1]-hardness results and present an FPT algorithm.
Uncertainty Quantification in Three Dimensional Natural Convection using Polynomial Chaos Expansion and Deep Neural Networks
Venue: arXiv cs.NA 2019
Year: 2019

This paper analyzes the effects of input uncertainties on the outputs of a three dimensional natural convection problem in a differentially heated cubical enclosure. Two different cases are considered for parameter uncertainty propagation and global sensitivity analysis. In case A, stochastic variation is introduced in the two non-dimensional parameters (Rayleigh and Prandtl numbers) with an assumption that the boundary temperature is uniform. Being a two dimensional stochastic problem, the polynomial chaos expansion (PCE) method is used as a surrogate model. Case B deals with non-uniform stochasticity in the boundary temperature. Instead of the traditional Gaussian process model with the Karhunen-Lo$\grave{e}$ve expansion, a novel approach is successfully implemented to model uncertainty in the boundary condition. The boundary is divided into multiple domains and the temperature imposed on each domain is assumed to be an independent and identically distributed (i.i.d) random variable. Deep neural networks are trained with the boundary temperatures as inputs and Nusselt number, internal temperature or velocities as outputs. The number of domains which is essentially the stochastic dimension is 4, 8, 16 or 32. Rigorous training and testing process shows that the neural network is able to approximate the outputs to a reasonable accuracy. For a high stochastic dimension such as 32, it is computationally expensive to fit the PCE. This paper demonstrates a novel way of using the deep neural network as a surrogate modeling method for uncertainty quantification with the number of simulations much fewer than that required for fitting the PCE, thus, saving the computational cost.
Assessing Generalization in Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Deep reinforcement learning (RL) has achieved breakthrough results on many tasks, but agents often fail to generalize beyond the environment they were trained in. As a result, deep RL algorithms that promote generalization are receiving increasing attention. However, works in this area use a wide variety of tasks and experimental setups for evaluation. The literature lacks a controlled assessment of the merits of different generalization schemes. Our aim is to catalyze community-wide progress on generalization in deep RL. To this end, we present a benchmark and experimental protocol, and conduct a systematic empirical study. Our framework contains a diverse set of environments, our methodology covers both in-distribution and out-of-distribution generalization, and our evaluation includes deep RL algorithms that specifically tackle generalization. Our key finding is that `vanilla' deep RL algorithms generalize better than specialized schemes that were proposed specifically to tackle generalization.
Generating new pictures in complex datasets with a simple neural network
Venue: arXiv cs.CV 2019
Year: 2019

We introduce a version of a variational auto-encoder (VAE), which can generate good perturbations of images, when trained on a complex dataset (in our experiments, CIFAR-10). The net is using only two latent generative dimensions per class, with uni-modal probability density. The price one has to pay for good generation is that not all training images are well reconstructed. An additional classifier is required to determine which training image is well reconstructed and generally the weights of training images. Only training images which are well reconstructed, can be perturbed. For good perturbations, we use the tentative empirical drifts of well reconstructed images. The construct is not predictive in the usual statistical sense.
A nearly optimal algorithm to decompose binary forms
Venue: arXiv cs.SC 2019
Year: 2019

Symmetric tensor decomposition is an important problem with applications in several areas for example signal processing, statistics, data analysis and computational neuroscience. It is equivalent to Waring's problem for homogeneous polynomials, that is to write a homogeneous polynomial in n variables of degree D as a sum of D-th powers of linear forms, using the minimal number of summands. This minimal number is called the rank of the polynomial/tensor. We focus on decomposing binary forms, a problem that corresponds to the decomposition of symmetric tensors of dimension 2 and order D. Under this formulation, the problem finds its roots in invariant theory where the decompositions are known as canonical forms. In this context many different algorithms were proposed. We introduce a superfast algorithm that improves the previous approaches with results from structured linear algebra. It achieves a softly linear arithmetic complexity bound. To the best of our knowledge, the previously known algorithms have at least quadratic complexity bounds. Our algorithm computes a symbolic decomposition in $O(M(D) log(D))$ arithmetic operations, where $M(D)$ is the complexity of multiplying two polynomials of degree D. It is deterministic when the decomposition is unique. When the decomposition is not unique, our algorithm is randomized. We present a Monte Carlo version of it and we show how to modify it to a Las Vegas one, within the same complexity. From the symbolic decomposition, we approximate the terms of the decomposition with an error of $2^{--$ε$}$ , in $O(D log^2(D) (log^2(D) + log($ε$)))$ arithmetic operations. We use results from Kaltofen and Yagati (1989) to bound the size of the representation of the coefficients involved in the decomposition and we bound the algebraic degree of the problem by min(rank, D -- rank + 1). We show that this bound can be tight. When the input polynomial has integer coefficients, our algorithm performs, up to poly-logarithmic factors, $O\_{bit} (D{\ell} + D^4 + D^3 $τ$)$ bit operations, where $$τ$$ is the maximum bitsize of the coefficients and $2^{--{\ell}}$ is the relative error of the terms in the decomposition.
Deep Learning for the Gaussian Wiretap Channel
Venue: arXiv cs.IT 2019
Year: 2019

End-to-end learning of communication systems with neural networks and particularly autoencoders is an emerging research direction which gained popularity in the last year. In this approach, neural networks learn to simultaneously optimize encoding and decoding functions to establish reliable message transmission. In this paper, this line of thinking is extended to communication scenarios in which an eavesdropper must further be kept ignorant about the communication. The secrecy of the transmission is achieved by utilizing a modified secure loss function based on cross-entropy which can be implemented with state-of-the-art machine-learning libraries. This secure loss function approach is applied in a Gaussian wiretap channel setup, for which it is shown that the neural network learns a trade-off between reliable communication and information secrecy by clustering learned constellations. As a result, an eavesdropper with higher noise cannot distinguish between the symbols anymore.
On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
Venue: arXiv cs.LG 2019
Year: 2019

Recent work has shown that it is possible to train deep neural networks that are provably robust to norm-bounded adversarial perturbations. Most of these methods are based on minimizing an upper bound on the worst-case loss over all possible adversarial perturbations. While these techniques show promise, they often result in difficult optimization procedures that remain hard to scale to larger networks. Through a comprehensive analysis, we show how a simple bounding technique, interval bound propagation (IBP), can be exploited to train large provably robust neural networks that beat the state-of-the-art in verified accuracy. While the upper bound computed by IBP can be quite weak for general networks, we demonstrate that an appropriate loss and clever hyper-parameter schedule allow the network to adapt such that the IBP bound is tight. This results in a fast and stable learning algorithm that outperforms more sophisticated methods and achieves state-of-the-art results on MNIST, CIFAR-10 and SVHN. It also allows us to train the largest model to be verified beyond vacuous bounds on a downscaled version of ImageNet.
Tracing Transactions Across Cryptocurrency Ledgers
Venue: arXiv cs.CR 2019
Year: 2019

One of the defining features of a cryptocurrency is that its ledger, containing all transactions that have evertaken place, is globally visible. As one consequenceof this degree of transparency, a long line of recent re-search has demonstrated that even in cryptocurrenciesthat are specifically designed to improve anonymity it is often possible to track money as it changes hands,and in some cases to de-anonymize users entirely. With the recent proliferation of alternative cryptocurrencies, however, it becomes relevant to ask not only whether ornot money can be traced as it moves within the ledgerof a single cryptocurrency, but if it can in fact be tracedas it moves across ledgers. This is especially pertinent given the rise in popularity of automated trading platforms such as ShapeShift, which make it effortless to carry out such cross-currency trades. In this paper, weuse data scraped from ShapeShift over a thirteen-monthperiod and the data from eight different blockchains to explore this question. Beyond developing new heuristics and creating new types of links across cryptocurrency ledgers, we also identify various patterns of cross-currency trades and of the general usage of these platforms, with the ultimate goal of understanding whetherthey serve a criminal or a profit-driven agenda.