OpenRaiser/Intern-Atlas huggingface dataset:
Constructing a provably adversarially-robust classifier from a high accuracy one
Venue: arXiv cs.LG 2019
Year: 2019

Modern machine learning models with very high accuracy have been shown to be vulnerable to small, adversarially chosen perturbations of the input. Given black-box access to a high-accuracy classifier $f$, we show how to construct a new classifier $g$ that has high accuracy and is also robust to adversarial $\ell_2$-bounded perturbations. Our algorithm builds upon the framework of \textit{randomized smoothing} that has been recently shown to outperform all previous defenses against $\ell_2$-bounded adversaries. Using techniques like random partitions and doubling dimension, we are able to bound the adversarial error of $g$ in terms of the optimum error. In this paper we focus on our conceptual contribution, but we do present two examples to illustrate our framework. We will argue that, under some assumptions, our bounds are optimal for these cases.
Network of Evolvable Neural Units: Evolving to Learn at a Synaptic Level
Venue: arXiv cs.LG 2019
Year: 2019

Although Deep Neural Networks have seen great success in recent years through various changes in overall architectures and optimization strategies, their fundamental underlying design remains largely unchanged. Computational neuroscience on the other hand provides more biologically realistic models of neural processing mechanisms, but they are still high level abstractions of the actual experimentally observed behaviour. Here a model is proposed that bridges Neuroscience, Machine Learning and Evolutionary Algorithms to evolve individual soma and synaptic compartment models of neurons in a scalable manner. Instead of attempting to manually derive models for all the observed complexity and diversity in neural processing, we propose an Evolvable Neural Unit (ENU) that can approximate the function of each individual neuron and synapse. We demonstrate that this type of unit can be evolved to mimic Integrate-And-Fire neurons and synaptic Spike-Timing-Dependent Plasticity. Additionally, by constructing a new type of neural network where each synapse and neuron is such an evolvable neural unit, we show it is possible to evolve an agent capable of learning to solve a T-maze environment task. This network independently discovers spiking dynamics and reinforcement type learning rules, opening up a new path towards biologically inspired artificial intelligence.
Game Theory based Joint Task Offloading and Resources Allocation Algorithm for Mobile Edge Computing
Venue: arXiv cs.NI 2019
Year: 2019

Mobile edge computing (MEC) has emerged for reducing energy consumption and latency by allowing mobile users to offload computationally intensive tasks to the MEC server. Due to the spectrum reuse in small cell network, the inter-cell interference has a great effect on MEC performances. In this paper, for reducing the energy consumption and latency of MEC, we propose a game theory based approach to join task offloading decision and resources allocation together in the MEC system. In this algorithm, the offloading decision, the CPU capacity adjustment, the transmission power control, and the network interference management of mobile users are regarded as a game. In this game, based on the best response strategy, each mobile user makes their own utility maximum rather than the utility of the whole system. We prove that this game is an exact potential game and the Nash equilibrium (NE) of this game exists. For reaching the NE, the best response approach is applied. We calculate the best response of these three variables. Moreover, we investigate the properties of this algorithm, including the convergence, the computational complexity, and the Price of anarchy (PoA). The theoretical analysis shows that the inter-cell interference affects on the performances of MEC greatly. The NE of this game is Pareto efficiency. Finally, we evaluate the performances of this algorithm by simulation. The simulation results illustrate that this algorithm is effective in improving the performances of the multi-user MEC system.
Alsvinn: A Fast multi-GPGPU finite volume solver with a strong emphasis on reproducibility
Venue: arXiv cs.MS 2019
Year: 2019

We present the Alsvinn simulator, a fast multi general purpose graphical processing unit (GPGPU) finite volume solver for hyperbolic conservation laws in multiple space dimensions. Alsvinn has native support for uncertainty quantifications, and exhibits excellent scaling on top tier compute clusters.
Pairwise Feedback for Data Programming
Venue: arXiv cs.LG 2019
Year: 2019

The scalability of the labeling process and the attainable quality of labels have become limiting factors for many applications of machine learning. The programmatic creation of labeled datasets via the synthesis of noisy heuristics provides a promising avenue to address this problem. We propose to improve modeling of latent class variables in the programmatic creation of labeled datasets by incorporating pairwise feedback into the process. We discuss the ease with which such pairwise feedback can be obtained or generated in many application domains. Our experiments show that even a small number of sources of pairwise feedback can substantially improve the quality of the posterior estimate of the latent class variable.
Adversarial Model Extraction on Graph Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Along with the advent of deep neural networks came various methods of exploitation, such as fooling the classifier or contaminating its training data. Another such attack is known as model extraction, where provided API access to some black box neural network, the adversary extracts the underlying model. This is done by querying the model in such a way that the underlying neural network provides enough information to the adversary to be reconstructed. While several works have achieved impressive results with neural network extraction in the propositional domain, this problem has not yet been considered over the relational domain, where data samples are no longer considered to be independent and identically distributed (iid). Graph Neural Networks (GNNs) are a popular deep learning framework to perform machine learning tasks over relational data. In this work, we formalize an instance of GNN extraction, present a solution with preliminary results, and discuss our assumptions and future directions.
LTLf Synthesis with Fairness and Stability Assumptions
Venue: arXiv cs.AI 2019
Year: 2019

In synthesis, assumptions are constraints on the environment that rule out certain environment behaviors. A key observation here is that even if we consider systems with LTLf goals on finite traces, environment assumptions need to be expressed over infinite traces, since accomplishing the agent goals may require an unbounded number of environment action. To solve synthesis with respect to finite-trace LTLf goals under infinite-trace assumptions, we could reduce the problem to LTL synthesis. Unfortunately, while synthesis in LTLf and in LTL have the same worst-case complexity (both 2EXPTIME-complete), the algorithms available for LTL synthesis are much more difficult in practice than those for LTLf synthesis. In this work we show that in interesting cases we can avoid such a detour to LTL synthesis and keep the simplicity of LTLf synthesis. Specifically, we develop a BDD-based fixpoint-based technique for handling basic forms of fairness and of stability assumptions. We show, empirically, that this technique performs much better than standard LTL synthesis.
PIRATE: A Blockchain-based Secure Framework of Distributed Machine Learning in 5G Networks
Venue: arXiv cs.DC 2019
Year: 2019

In the fifth-generation (5G) networks and the beyond, communication latency and network bandwidth will be no more bottleneck to mobile users. Thus, almost every mobile device can participate in the distributed learning. That is, the availability issue of distributed learning can be eliminated. However, the model safety will become a challenge. This is because the distributed learning system is prone to suffering from byzantine attacks during the stages of updating model parameters and aggregating gradients amongst multiple learning participants. Therefore, to provide the byzantine-resilience for distributed learning in 5G era, this article proposes a secure computing framework based on the sharding-technique of blockchain, namely PIRATE. A case-study shows how the proposed PIRATE contributes to the distributed learning. Finally, we also envision some open issues and challenges based on the proposed byzantine-resilient learning framework.
Constructing the F-Graph with a Symmetric Constraint for Subspace Clustering
Venue: arXiv cs.CV 2019
Year: 2019

Based on further studying the low-rank subspace clustering (LRSC) and L2-graph subspace clustering algorithms, we propose a F-graph subspace clustering algorithm with a symmetric constraint (FSSC), which constructs a new objective function with a symmetric constraint basing on F-norm, whose the most significant advantage is to obtain a closed-form solution of the coefficient matrix. Then, take the absolute value of each element of the coefficient matrix, and retain the k largest coefficients per column, set the other elements to 0, to get a new coefficient matrix. Finally, FSSC performs spectral clustering over the new coefficient matrix. The experimental results on face clustering and motion segmentation show FSSC algorithm can not only obviously reduce the running time, but also achieve higher accuracy compared with the state-of-the-art representation-based subspace clustering algorithms, which verifies that the FSSC algorithm is efficacious and feasible.
Asynchronous Federated Learning with Differential Privacy for Edge Intelligence
Venue: arXiv cs.LG 2019
Year: 2019

Federated learning has been showing as a promising approach in paving the last mile of artificial intelligence, due to its great potential of solving the data isolation problem in large scale machine learning. Particularly, with consideration of the heterogeneity in practical edge computing systems, asynchronous edge-cloud collaboration based federated learning can further improve the learning efficiency by significantly reducing the straggler effect. Despite no raw data sharing, the open architecture and extensive collaborations of asynchronous federated learning (AFL) still give some malicious participants great opportunities to infer other parties' training data, thus leading to serious concerns of privacy. To achieve a rigorous privacy guarantee with high utility, we investigate to secure asynchronous edge-cloud collaborative federated learning with differential privacy, focusing on the impacts of differential privacy on model convergence of AFL. Formally, we give the first analysis on the model convergence of AFL under DP and propose a multi-stage adjustable private algorithm (MAPA) to improve the trade-off between model utility and privacy by dynamically adjusting both the noise scale and the learning rate. Through extensive simulations and real-world experiments with an edge-could testbed, we demonstrate that MAPA significantly improves both the model accuracy and convergence speed with sufficient privacy guarantee.
A Routing Metric for Inter-flow Interference-aware Flying Multi-hop Networks
Venue: arXiv cs.NI 2019
Year: 2019

The growing demand for broadband communications anytime, anywhere has paved the way to the usage of Unmanned Aerial Vehicles (UAVs) for providing Internet access in areas without network infrastructure and enhancing the performance of existing networks. However, the usage of Flying Multi-hop Networks (FMNs) in such scenarios brings up significant challenges concerning network routing, in order to permanently provide the Quality of Service expected by the users. The problem is exacerbated in crowded events, where the FMN may be formed by many UAVs to address the traffic demand, causing inter-flow interference within the FMN. Typically, estimating inter-flow interference is not straightforward and requires the exchange of probe packets, thus increasing network overhead. The main contribution of this paper is an inter-flow interference-aware routing metric, named I2R, designed for centralized routing in FMNs with controllable topology. I2R does not require any control packets and enables the configuration of paths with minimal Euclidean distance formed by UAVs with the lowest number of neighbors in carrier-sense range, thus minimizing inter-flow interference in the FMN. Simulation results show the I2R superior performance, with significant gains in terms of throughput and end-to-end delay, when compared with state of the art routing metrics.
Embedded Constrained Feature Construction for High-Energy Physics Data Classification
Venue: arXiv cs.LG 2019
Year: 2019

Before any publication, data analysis of high-energy physics experiments must be validated. This validation is granted only if a perfect understanding of the data and the analysis process is demonstrated. Therefore, physicists prefer using transparent machine learning algorithms whose performances highly rely on the suitability of the provided input features. To transform the feature space, feature construction aims at automatically generating new relevant features. Whereas most of previous works in this area perform the feature construction prior to the model training, we propose here a general framework to embed a feature construction technique adapted to the constraints of high-energy physics in the induction of tree-based models. Experiments on two high-energy physics datasets confirm that a significant gain is obtained on the classification scores, while limiting the number of built features. Since the features are built to be interpretable, the whole model is transparent and readable.
Sim-to-Real Domain Adaptation For High Energy Physics
Venue: arXiv cs.LG 2019
Year: 2019

Particle physics or High Energy Physics (HEP) studies the elementary constituents of matter and their interactions with each other. Machine Learning (ML) has played an important role in HEP analysis and has proven extremely successful in this area. Usually, the ML algorithms are trained on numerical simulations of the experimental setup and then applied to the real experimental data. However, any discrepancy between the simulation and real data may lead to dramatic consequences concerning the performances of the algorithm on real data. In this paper, we present an application of domain adaptation using a Domain Adversarial Neural Network trained on public HEP data. We demonstrate the success of this approach to achieve sim-to-real transfer and ensure the consistency of the ML algorithms performances on real and simulated HEP datasets.
Querying Linked Data: An Experimental Evaluation of State-of-the-Art Interfaces
Venue: arXiv cs.DB 2019
Year: 2019

The adoption of Semantic Web technologies, and in particular the Open Data initiative, has contributed to the steady growth of the number of datasets and triples accessible on the Web. Most commonly, queries over RDF data are evaluated over SPARQL endpoints. Recently, however, alternatives such as TPF have been proposed with the goal of shifting query processing load from the server running the SPARQL endpoint towards the client that issued the query. Although these interfaces have been evaluated against standard benchmarks and testbeds that showed their benefits over previous work in general, a fine-granular evaluation of what types of queries exploit the strengths of the different available interfaces has never been done. In this paper, we present the results of our in-depth evaluation of existing RDF interfaces. In addition, we also examine the influence of the backend on the performance of these interfaces. Using representative and diverse query loads based on the query log of a public SPARQL endpoint, we stress test the different interfaces and backends and identify their strengths and weaknesses.
Application of Word2vec in Phoneme Recognition
Venue: arXiv cs.CL 2019
Year: 2019

In this paper, we present how to hybridize a Word2vec model and an attention-based end-to-end speech recognition model. We build a phoneme recognition system based on Listen, Attend and Spell model. And the phoneme recognition model uses a word2vec model to initialize the embedding matrix for the improvement of the performance, which can increase the distance among the phoneme vectors. At the same time, in order to solve the problem of overfitting in the 61 phoneme recognition model on TIMIT dataset, we propose a new training method. A 61-39 phoneme mapping comparison table is used to inverse map the phonemes of the dataset to generate more 61 phoneme training data. At the end of training, replace the dataset with a standard dataset for corrective training. Our model can achieve the best result under the TIMIT dataset which is 16.5% PER (Phoneme Error Rate).
A UAV-Mounted Free Space Optical Communication: Trajectory Optimization for Flight Time
Venue: arXiv cs.IT 2019
Year: 2019

In this work, we address the trajectory optimization of a fixed-wing unmanned aerial vehicle (UAV) using free space optical communication (FSOC). Here, we focus on maximizing the flight time of the UAV by considering practical constraints for wireless UAV communication, including limited propulsion energy and required data rates. We find optimized trajectories in various atmospheric environments (e.g., moderate-fog and heavy-fog conditions), while also considering the channel characteristics of FSOC. In addition to maximizing the flight time, we consider the energy efficiency maximization and operation-time minimization problem to find the suboptimal solutions required to meet those constraints. Furthermore, we introduce a low-complexity approach to the proposed framework. In order to address the optimization problem, we conduct a bisection method and sequential programming and introduce a new feasibility check algorithm. Although our design considers suboptimal solutions owing to the nonconvexity of the problems, our simulations indicate that the proposed scheme exhibits a gain of approximately 44.12\% in terms of service time when compared to the conventional scheme.
Progressive VAE Training on Highly Sparse and Imbalanced Data
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we present a novel approach for training a Variational Autoencoder (VAE) on a highly imbalanced data set. The proposed training of a high-resolution VAE model begins with the training of a low-resolution core model, which can be successfully trained on imbalanced data set. In subsequent training steps, new convolutional, upsampling, deconvolutional, and downsampling layers are iteratively attached to the model. In each iteration, the additional layers are trained based on the intermediate pretrained model - a result of previous training iterations. Thus, the resolution of the model is progressively increased up to the required resolution level. In this paper, the progressive VAE training is exploited for learning a latent representation with imbalanced, highly sparse data sets and, consequently, generating routes in a constrained 2D space. Routing problems (e.g., vehicle routing problem, travelling salesman problem, and arc routing) are of special significance in many modern applications (e.g., route planning, network maintenance, developing high-performance nanoelectronic systems, and others) and typically associated with sparse imbalanced data. In this paper, the critical problem of routing billions of components in nanoelectronic devices is considered. The proposed approach exhibits a significant training speedup as compared with state-of-the-art existing VAE training methods, while generating expected image outputs from unseen input data. Furthermore, the final progressive VAE models exhibit much more precise output representation, than the Generative Adversarial Network (GAN) models trained with comparable training time. The proposed method is expected to be applicable to a wide range of applications, including but not limited image impainting, sentence interpolation, and semi-supervised learning.
CoulGAT: An Experiment on Interpretability of Graph Attention Networks
Venue: arXiv cs.LG 2019
Year: 2019

We present an attention mechanism inspired from definition of screened Coulomb potential. This attention mechanism was used to interpret the Graph Attention (GAT) model layers and training dataset by using a flexible and scalable framework (CoulGAT) developed for this purpose. Using CoulGAT, a forest of plain and resnet models were trained and characterized using this attention mechanism against CHAMPS dataset. The learnable variables of the attention mechanism are used to extract node-node and node-feature interactions to define an empirical standard model for the graph structure and hidden layer. This representation of graph and hidden layers can be used as a tool to compare different models, optimize hidden layers and extract a compact definition of graph structure of the dataset.
Relational Mimic for Visual Adversarial Imitation Learning
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we introduce a new method for imitation learning from video demonstrations. Our method, Relational Mimic (RM), improves on previous visual imitation learning methods by combining generative adversarial networks and relational learning. RM is flexible and can be used in conjunction with other recent advances in generative adversarial imitation learning to better address the need for more robust and sample-efficient approaches. In addition, we introduce a new neural network architecture that improves upon the previous state-of-the-art in reinforcement learning and illustrate how increasing the relational reasoning capabilities of the agent enables the latter to achieve increasingly higher performance in a challenging locomotion task with pixel inputs. Finally, we study the effects and contributions of relational learning in policy evaluation, policy improvement and reward learning through ablation studies.
A Web Page Classifier Library Based on Random Image Content Analysis Using Deep Learning
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we present a methodology and the corresponding Python library 1 for the classification of webpages. Our method retrieves a fixed number of images from a given webpage, and based on them classifies the webpage into a set of established classes with a given probability. The library trains a random forest model build upon the features extracted from images by a pre-trained deep network. The implementation is tested by recognizing weapon class webpages in a curated list of 3859 websites. The results show that the best method of classifying a webpage into the studies classes is to assign the class according to the maximum probability of any image belonging to this (weapon) class being above the threshold, across all the retrieved images. Further research explores the possibilities for the developed methodology to also apply in image classification for healthcare applications.