OpenRaiser/Intern-Atlas huggingface dataset:
A New Perspective on Machine Learning: How to do Perfect Supervised Learning
Venue: arXiv cs.LG 2019
Year: 2019

In this work, we introduce the concept of bandlimiting into the theory of machine learning because all physical processes are bandlimited by nature, including real-world machine learning tasks. After the bandlimiting constraint is taken into account, our theoretical analysis has shown that all practical machine learning tasks are asymptotically solvable in a perfect sense. Furthermore, the key towards this solvability almost solely relies on two factors: i) a sufficiently large amount of training samples beyond a threshold determined by a difficulty measurement of the underlying task; ii) a sufficiently complex and bandlimited model. Moreover, for some special cases, we have derived new error bounds for perfect learning, which can quantify the difficulty of learning. These generalization bounds are not only asymptotically convergent but also irrelevant to model complexity. Our new results on generalization have provided a new perspective to explain the recent successes of large-scale supervised learning using complex models like neural networks.
Influence Minimization Under Budget and Matroid Constraints: Extended Version
Venue: arXiv cs.SI 2019
Year: 2019

Recently, online social networks have become major battlegrounds for political campaigns, viral marketing, and the dissemination of news. As a consequence, ''bad actors'' are increasingly exploiting these platforms, becoming a key challenge for their administrators, businesses and the society in general. The spread of fake news is a classical example of the abuse of social networks by these actors. While some have advocated for stricter policies to control the spread of misinformation in social networks, this often happens in detriment of their democratic and organic structure. In this paper we study how to limit the influence of a target set of users in a network via the removal of a few edges. The idea is to control the diffusion processes while minimizing the amount of disturbance in the network structure. We formulate the influence limitation problem in a data-driven fashion, by taking into account past propagation traces. Moreover, we consider two types of constraints over the set of edge removals, a budget constraint and also a, more general, set of matroid constraints. These problems lead to interesting challenges in terms of algorithm design. For instance, we are able to show that influence limitation is APX-hard and propose deterministic and probabilistic approximation algorithms for the budgeted and matroid version of the problem, respectively. Our experiments show that the proposed solutions outperform the baselines by up to 40%.
Risk-Aware Active Inverse Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Active learning from demonstration allows a robot to query a human for specific types of input to achieve efficient learning. Existing work has explored a variety of active query strategies; however, to our knowledge, none of these strategies directly minimize the performance risk of the policy the robot is learning. Utilizing recent advances in performance bounds for inverse reinforcement learning, we propose a risk-aware active inverse reinforcement learning algorithm that focuses active queries on areas of the state space with the potential for large generalization error. We show that risk-aware active learning outperforms standard active IRL approaches on gridworld, simulated driving, and table setting tasks, while also providing a performance-based stopping criterion that allows a robot to know when it has received enough demonstrations to safely perform a task.
Solar-Sail Trajectory Design for Multiple Near Earth Asteroid Exploration Based on Deep Neural Networks
Venue: arXiv cs.CE 2019
Year: 2019

In the preliminary trajectory design of the multi-target rendezvous problem, a model that can quickly estimate the cost of the orbital transfer is essential. The estimation of the transfer time using solar sail between two arbitrary orbits is difficult and usually requires to solve an optimal control problem. Inspired by the successful applications of the deep neural network in nonlinear regression, this work explores the possibility and effectiveness of mapping the transfer time for solar sail from the orbital characteristics using the deep neural network. Furthermore, the Monte Carlo Tree Search method is investigated and used to search the optimal sequence considering a multi-asteroid exploration problem. The obtained sequences from preliminary design will be solved and verified by sequentially solving the optimal control problem. Two examples of different application backgrounds validate the effectiveness of the proposed approach.
Age Optimal Information Gathering and Dissemination on Graphs
Venue: arXiv cs.IT 2019
Year: 2019

We consider the problem of timely exchange of updates between a central station and a set of ground terminals $V$, via a mobile agent that traverses across the ground terminals along a mobility graph $G = (V, E)$. We design the trajectory of the mobile agent to minimize peak and average age of information (AoI), two newly proposed metrics for measuring timeliness of information. We consider randomized trajectories, in which the mobile agent travels from terminal $i$ to terminal $j$ with probability $P_{i,j}$. For the information gathering problem, we show that a randomized trajectory is peak age optimal and factor-$8\mathcal{H}$ average age optimal, where $\mathcal{H}$ is the mixing time of the randomized trajectory on the mobility graph $G$. We also show that the average age minimization problem is NP-hard. For the information dissemination problem, we prove that the same randomized trajectory is factor-$O(\mathcal{H})$ peak and average age optimal. Moreover, we propose an age-based trajectory, which utilizes information about current age at terminals, and show that it is factor-$2$ average age optimal in a symmetric setting.
Data Masking with Privacy Guarantees
Venue: arXiv cs.LG 2019
Year: 2019

We study the problem of data release with privacy, where data is made available with privacy guarantees while keeping the usability of the data as high as possible --- this is important in health-care and other domains with sensitive data. In particular, we propose a method of masking the private data with privacy guarantee while ensuring that a classifier trained on the masked data is similar to the classifier trained on the original data, to maintain usability. We analyze the theoretical risks of the proposed method and the traditional input perturbation method. Results show that the proposed method achieves lower risk compared to the input perturbation, especially when the number of training samples gets large. We illustrate the effectiveness of the proposed method of data masking for privacy-sensitive learning on $12$ benchmark datasets.
Uncertainty-Based Out-of-Distribution Detection in Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

We consider the problem of detecting out-of-distribution (OOD) samples in deep reinforcement learning. In a value based reinforcement learning setting, we propose to use uncertainty estimation techniques directly on the agent's value estimating neural network to detect OOD samples. The focus of our work lies in analyzing the suitability of approximate Bayesian inference methods and related ensembling techniques that generate uncertainty estimates. Although prior work has shown that dropout-based variational inference techniques and bootstrap-based approaches can be used to model epistemic uncertainty, the suitability for detecting OOD samples in deep reinforcement learning remains an open question. Our results show that uncertainty estimation can be used to differentiate in- from out-of-distribution samples. Over the complete training process of the reinforcement learning agents, bootstrap-based approaches tend to produce more reliable epistemic uncertainty estimates, when compared to dropout-based approaches.
Rate matching for polar codes based on binary domination
Venue: arXiv cs.IT 2019
Year: 2019

In this paper, we investigate the fundamentals of puncturing and shortening for polar codes, based on binary domination which plays a key role in polar code construction. We first prove that the orders of encoder input bits to be made incapable (by puncturing) or to be shortened are governed by binary domination. In particular, we show that binary domination completely determines incapable or shortened bit patterns for polar codes, and that all the possible incapable or shortened bit patterns can be identified. We then present the patterns of the corresponding encoder output bits to be punctured or fixed, when the incapable or shortened bits are given. We also demonstrate that the order and the pattern of puncturing and shortening for polar codes can be aligned. In the previous work on the rate matching for polar codes, puncturing of encoder output bits begins from a low-indexed bit, while shortening starts from a high-indexed bit. Unlike such a conventional approach, we show that encoder output bits can be punctured from high-indexed bits, while keeping the incapable bit pattern exactly the same. This makes it possible to design a unified circular-buffer rate matching (CB-RM) scheme that includes puncturing, shortening, and repetition.
Tutorial on UAV: A Blue Sky View on Wireless Communication
Venue: arXiv cs.NI 2019
Year: 2019

The growing use of Unmanned Aerial Vehicles (UAVs) for various applications requires ubiquitous and reliable connectivity for safe control and data exchange between these devices and ground terminals. Depending on the application, UAV-mounted wireless equipment can either be an aerial user equipment (AUE) that co-exists with the terrestrial users, or it can be a part of wireless infrastructure providing a range of services to the ground users. For instance, AUE can be used for real-time search and rescue and Aerial Base Station (ABS) can enhance coverage, capacity and energy efficiency of wireless networks. In both cases, UAV-based solutions are scalable, mobile, fast to deploy. However, several technical challenges have to be addressed. In this work, we present a tutorial on wireless communication with UAVs, taking into account a wide range of potential applications. The main goal of this work is to provide a complete overview of the main scenarios (AUE and ABS), channel and performance models, compare them, and discuss open research points. This work gives a comprehensive overview of the research done until now and depicts a comprehensive picture to foster new ideas and solutions while avoiding duplication of past work. We start by discussing the open challenges of wireless communication with UAVs. To give answers to the posed questions, we focus on the UAV communication basics, mainly providing the necessary channel modeling background and giving guidelines on how various channel models should be used. Next, theoretical, simulation- and measurement-based approaches, to address the key challenges for AUE usage, are presented. Moreover, in this work, we aim to provide a comprehensive overview on how UAV-mounted equipment can be used as a part of a communication network. Based on the theoretical analysis, we show how various network parameters can be optimized.
Complexity Bounds for the Controllability of Temporal Networks with Conditions, Disjunctions, and Uncertainty
Venue: arXiv cs.AI 2019
Year: 2019

In temporal planning, many different temporal network formalisms are used to model real world situations. Each of these formalisms has different features which affect how easy it is to determine whether the underlying network of temporal constraints is consistent. While many of the simpler models have been well-studied from a computational complexity perspective, the algorithms developed for advanced models which combine features have very loose complexity bounds. In this paper, we provide tight completeness bounds for strong, weak, and dynamic controllability checking of temporal networks that have conditions, disjunctions, and temporal uncertainty. Our work exposes some of the subtle differences between these different structures and, remarkably, establishes a guarantee that all of these problems are computable in PSPACE.
Fusion Strategies for Learning User Embeddings with Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

Growing amounts of online user data motivate the need for automated processing techniques. In case of user ratings, one interesting option is to use neural networks for learning to predict ratings given an item and a user. While training for prediction, such an approach at the same time learns to map each user to a vector, a so-called user embedding. Such embeddings can for example be valuable for estimating user similarity. However, there are various ways how item and user information can be combined in neural networks, and it is unclear how the way of combining affects the resulting embeddings. In this paper, we run an experiment on movie ratings data, where we analyze the effect on embedding quality caused by several fusion strategies in neural networks. For evaluating embedding quality, we propose a novel measure, Pair-Distance Correlation, which quantifies the condition that similar users should have similar embedding vectors. We find that the fusion strategy affects results in terms of both prediction performance and embedding quality. Surprisingly, we find that prediction performance not necessarily reflects embedding quality. This suggests that if embeddings are of interest, the common tendency to select models based on their prediction ability should be reconsidered.
Robust and High Performance Face Detector
Venue: arXiv cs.CV 2019
Year: 2019

In recent years, face detection has experienced significant performance improvement with the boost of deep convolutional neural networks. In this report, we reimplement the state-of-the-art detector SRN and apply some tricks proposed in the recent literatures to obtain an extremely strong face detector, named VIM-FD. In specific, we exploit more powerful backbone network like DenseNet-121, revisit the data augmentation based on data-anchor-sampling proposed in PyramidBox, and use the max-in-out label and anchor matching strategy in SFD. In addition, we also introduce the attention mechanism to provide additional supervision. Over the most popular and challenging face detection benchmark, i.e., WIDER FACE, the proposed VIM-FD achieves state-of-the-art performance.
Service Rate Region of Content Access from Erasure Coded Storage
Venue: arXiv cs.IT 2019
Year: 2019

We consider storage systems in which $K$ files are stored over $N$ nodes. A node may be systematic for a particular file in the sense that access to it gives access to the file. Alternatively, a node may be coded, meaning that it gives access to a particular file only when combined with other nodes (which may be coded or systematic). Requests for file $f_k$ arrive at rate $λ_k$, and we are interested in the rate that can be served by a particular system. In this paper, we determine the set of request arrival rates for the a $3$-file coded storage system. We also provide an algorithm to maximize the rate of requests served for file $K$ given $λ_1,\dots, λ_{K-1}$ in a general $K$-file case.
GILT: Generating Images from Long Text
Venue: arXiv cs.CV 2019
Year: 2019

Creating an image reflecting the content of a long text is a complex process that requires a sense of creativity. For example, creating a book cover or a movie poster based on their summary or a food image based on its recipe. In this paper we present the new task of generating images from long text that does not describe the visual content of the image directly. For this, we build a system for generating high-resolution 256 $\times$ 256 images of food conditioned on their recipes. The relation between the recipe text (without its title) to the visual content of the image is vague, and the textual structure of recipes is complex, consisting of two sections (ingredients and instructions) both containing multiple sentences. We used the recipe1M dataset to train and evaluate our model that is based on a the StackGAN-v2 architecture.
SNRA: A Spintronic Neuromorphic Reconfigurable Array for In-Circuit Training and Evaluation of Deep Belief Networks
Venue: arXiv cs.ET 2019
Year: 2019

In this paper, a spintronic neuromorphic reconfigurable Array (SNRA) is developed to fuse together power-efficient probabilistic and in-field programmable deterministic computing during both training and evaluation phases of restricted Boltzmann machines (RBMs). First, probabilistic spin logic devices are used to develop an RBM realization which is adapted to construct deep belief networks (DBNs) having one to three hidden layers of size 10 to 800 neurons each. Second, we design a hardware implementation for the contrastive divergence (CD) algorithm using a four-state finite state machine capable of unsupervised training in N+3 clocks where N denotes the number of neurons in each RBM. The functionality of our proposed CD hardware implementation is validated using ModelSim simulations. We synthesize the developed Verilog HDL implementation of our proposed test/train control circuitry for various DBN topologies where the maximal RBM dimensions yield resource utilization ranging from 51 to 2,421 lookup tables (LUTs). Next, we leverage spin Hall effect (SHE)-magnetic tunnel junction (MTJ) based non-volatile LUTs circuits as an alternative for static random access memory (SRAM)-based LUTs storing the deterministic logic configuration to form a reconfigurable fabric. Finally, we compare the performance of our proposed SNRA with SRAM-based configurable fabrics focusing on the area and power consumption induced by the LUTs used to implement both CD and evaluation modes. The results obtained indicate more than 80% reduction in combined dynamic and static power dissipation, while achieving at least 50% reduction in device count.
Learning with Collaborative Neural Network Group by Reflection
Venue: arXiv cs.LG 2019
Year: 2019

For the present engineering of neural systems, the preparing of extensive scale learning undertakings generally not just requires a huge neural system with a mind boggling preparing process yet additionally troublesome discover a clarification for genuine applications. In this paper, we might want to present the Collaborative Neural Network Group (CNNG). CNNG is a progression of neural systems that work cooperatively to deal with various errands independently in a similar learning framework. It is advanced from a solitary neural system by reflection. Along these lines, in light of various circumstances removed by the calculation, the CNNG can perform diverse techniques when handling the information. The examples of chose methodology can be seen by human to make profound adapting more reasonable. In our execution, the CNNG is joined by a few moderately little neural systems. We give a progression of examinations to assess the execution of CNNG contrasted with other learning strategies. The CNNG is able to get a higher accuracy with a much lower training cost. We can reduce the error rate by 74.5% and reached the accuracy of 99.45% in MNIST with three feedforward networks (4 layers) in one training epoch.
Unseen Object Segmentation in Videos via Transferable Representations
Venue: arXiv cs.CV 2019
Year: 2019

In order to learn object segmentation models in videos, conventional methods require a large amount of pixel-wise ground truth annotations. However, collecting such supervised data is time-consuming and labor-intensive. In this paper, we exploit existing annotations in source images and transfer such visual information to segment videos with unseen object categories. Without using any annotations in the target video, we propose a method to jointly mine useful segments and learn feature representations that better adapt to the target frames. The entire process is decomposed into two tasks: 1) solving a submodular function for selecting object-like segments, and 2) learning a CNN model with a transferable module for adapting seen categories in the source domain to the unseen target video. We present an iterative update scheme between two tasks to self-learn the final solution for object segmentation. Experimental results on numerous benchmark datasets show that the proposed method performs favorably against the state-of-the-art algorithms.
Face Recognition System
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning is one of the new and important branches in machine learning. Deep learning refers to a set of algorithms that solve various problems such as images and texts by using various machine learning algorithms in multi-layer neural networks. Deep learning can be classified as a neural network from the general category, but there are many changes in the concrete realization. At the core of deep learning is feature learning, which is designed to obtain hierarchical information through hierarchical networks, so as to solve the important problems that previously required artificial design features. Deep Learning is a framework that contains several important algorithms. For different applications (images, voice, text), you need to use different network models to achieve better results. With the development of deep learning and the introduction of deep convolutional neural networks, the accuracy and speed of face recognition have made great strides. However, as we said above, the results from different networks and models are very different. In this paper, facial features are extracted by merging and comparing multiple models, and then a deep neural network is constructed to train and construct the combined features. In this way, the advantages of multiple models can be combined to mention the recognition accuracy. After getting a model with high accuracy, we build a product model. This article compares the pure-client model with the server-client model, analyzes the pros and cons of the two models, and analyzes the various commercial products that are required for the server-client model.
Differentially Private Generative Adversarial Networks for Time Series, Continuous, and Discrete Open Data
Venue: arXiv cs.CR 2019
Year: 2019

Open data plays a fundamental role in the 21th century by stimulating economic growth and by enabling more transparent and inclusive societies. However, it is always difficult to create new high-quality datasets with the required privacy guarantees for many use cases. This paper aims at creating a framework for releasing new open data while protecting the individuality of the users through a strict definition of privacy called differential privacy. Unlike previous work, this paper provides a framework for privacy preserving data publishing that can be easily adapted to different use cases, from the generation of time-series to continuous data, and discrete data; no previous work has focused on the later class. Indeed, many use cases expose discrete data or at least a combination between categorical and numerical values. Thanks to the latest developments in deep learning and generative models, it is now possible to model rich-semantic data maintaining both the original distribution of the features and the correlations between them. The output of this framework is a deep network, namely a generator, able to create new data on demand. We demonstrate the efficiency of our approach on real datasets from the French public administration and classic benchmark datasets.
Faster parameterized algorithm for pumpkin vertex deletion set
Venue: arXiv cs.DS 2019
Year: 2019

A directed graph $G$ is called a pumpkin if $G$ is a union of induced paths with a common start vertex $s$ and a common end vertex $t$, and the internal vertices of every two paths are disjoint. We give an algorithm that given a directed graph $G$ and an integer $k$, decides whether a pumpkin can be obtained from $G$ by deleting at most $k$ vertices. The algorithm runs in $O^*(2^k)$ time.