OpenRaiser/Intern-Atlas huggingface dataset:
Performance Analysis of Large Intelligent Surfaces (LISs): Asymptotic Data Rate and Channel Hardening Effects
Venue: arXiv cs.IT 2019
Year: 2019

The concept of a large intelligent surface (LIS) has recently emerged as a promising wireless communication paradigm that can exploit the entire surface of man-made structures for transmitting and receiving information. An LIS is expected to go beyond massive multiple-input multiple-output (MIMO) system, insofar as the desired channel can be modeled as a perfect line-of-sight. To understand the fundamental performance benefits, it is imperative to analyze its achievable data rate, under practical LIS environments and limitations. In this paper, an asymptotic analysis of the uplink data rate in an LIS-based large antenna-array system is presented. In particular, the asymptotic LIS rate is derived in a practical wireless environment where the estimated channel on LIS is subject to estimation errors and interference channels are spatially correlated Rician fading channels. Moreover, the occurrence of the channel hardening effect is analyzed and the performance bound is asymptotically derived for the considered LIS system. The analytical asymptotic results are then shown to be in close agreement with the exact mutual information as the numbers of antennas and devices increases without bounds. Moreover, the derived ergodic rates show that noise and interference from estimation errors and the non-line-of-sight path become negligible as the number of antennas increases. Simulation results show that an LIS can achieve a performance that is comparable to conventional massive MIMO with improved reliability and a significantly reduced area for antenna deployment.
Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors
Venue: arXiv cs.CV 2019
Year: 2019

Cloud detection is an important preprocessing step for the precise application of optical satellite imagery. In this paper, we propose a deep learning based cloud detection method named multi-scale convolutional feature fusion (MSCFF) for remote sensing images of different sensors. In the network architecture of MSCFF, the symmetric encoder-decoder module, which provides both local and global context by densifying feature maps with trainable convolutional filter banks, is utilized to extract multi-scale and high-level spatial features. The feature maps of multiple scales are then up-sampled and concatenated, and a novel multi-scale feature fusion module is designed to fuse the features of different scales for the output. The two output feature maps of the network are cloud and cloud shadow maps, which are in turn fed to binary classifiers outside the model to obtain the final cloud and cloud shadow mask. The MSCFF method was validated on hundreds of globally distributed optical satellite images, with spatial resolutions ranging from 0.5 to 50 m, including Landsat-5/7/8, Gaofen-1/2/4, Sentinel-2, Ziyuan-3, CBERS-04, Huanjing-1, and collected high-resolution images exported from Google Earth. The experimental results show that MSCFF achieves a higher accuracy than the traditional rule-based cloud detection methods and the state-of-the-art deep learning models, especially in bright surface covered areas. The effectiveness of MSCFF means that it has great promise for the practical application of cloud detection for multiple types of medium and high-resolution remote sensing images. Our established global high-resolution cloud detection validation dataset has been made available online.
Enhancing Power System Cyber-Security with Systematic Two-Stage Detection Strategy
Venue: arXiv cs.SY 2019
Year: 2019

State estimation estimates the system condition in real-time and provides a base case for other energy management system (EMS) applications including real-time contingency analysis and security-constrained economic dispatch. Recent work in the literature shows malicious cyber-attack can inject false measurements that bypass traditional bad data detection in state estimation and cause actual overloads. Thus, it is very important to detect such cyber-attack. In this paper, multiple metrics are proposed to monitor abnormal load deviations and suspicious branch flow changes. A systematic two-stage approach is proposed to detect false data injection (FDI) cyber-attack. The first stage determines whether the system is under attack while the second stage identifies the target branch. Numerical simulations verify that FDI can cause severe system violations and demonstrate the effectiveness of the proposed two-stage FDI detection (FDID) method. It is concluded that the proposed FDID approach can efficiently detect FDI cyber-attack and identify the target branch, which will substantially improve operators situation awareness in real-time.
Varifocal-Net: A Chromosome Classification Approach using Deep Convolutional Networks
Venue: arXiv cs.CV 2019
Year: 2019

Chromosome classification is critical for karyotyping in abnormality diagnosis. To expedite the diagnosis, we present a novel method named Varifocal-Net for simultaneous classification of chromosome's type and polarity using deep convolutional networks. The approach consists of one global-scale network (G-Net) and one local-scale network (L-Net). It follows three stages. The first stage is to learn both global and local features. We extract global features and detect finer local regions via the G-Net. By proposing a varifocal mechanism, we zoom into local parts and extract local features via the L-Net. Residual learning and multi-task learning strategies are utilized to promote high-level feature extraction. The detection of discriminative local parts is fulfilled by a localization subnet of the G-Net, whose training process involves both supervised and weakly-supervised learning. The second stage is to build two multi-layer perceptron classifiers that exploit features of both two scales to boost classification performance. The third stage is to introduce a dispatch strategy of assigning each chromosome to a type within each patient case, by utilizing the domain knowledge of karyotyping. Evaluation results from 1909 karyotyping cases showed that the proposed Varifocal-Net achieved the highest accuracy per patient case (%) 99.2 for both type and polarity tasks. It outperformed state-of-the-art methods, demonstrating the effectiveness of our varifocal mechanism, multi-scale feature ensemble, and dispatch strategy. The proposed method has been applied to assist practical karyotype diagnosis.
An Efficient Fault Tolerant Workflow Scheduling Approach using Replication Heuristics and Checkpointing in the Cloud
Venue: arXiv cs.DC 2019
Year: 2019

Scientific workflows have been predominantly used for complex and large scale data analysis and scientific computation/automation and the need for robust workflow scheduling techniques has grown considerably. But, most of the existing workflow scheduling algorithms do not provide the required reliability and robustness. In this paper, a new fault tolerant workflow scheduling algorithm that learns replication heuristics in an unsupervised manner has been proposed. Furthermore, the use of light weight synchronized checkpointing enables efficient resubmission of failed tasks and ensures workflow completion even in precarious environments. The proposed technique improves upon metrics like Resource Wastage and Resource Usage in comparison to the Replicate-All algorithm, while maintaining an acceptable increase in Makespan as compared to the vanilla Heterogeneous Earliest Finish Time (HEFT).
Factorized Machine Self-Confidence for Decision-Making Agents
Venue: arXiv cs.LG 2019
Year: 2019

Algorithmic assurances from advanced autonomous systems assist human users in understanding, trusting, and using such systems appropriately. Designing these systems with the capacity of assessing their own capabilities is one approach to creating an algorithmic assurance. The idea of `machine self-confidence' is introduced for autonomous systems. Using a factorization based framework for self-confidence assessment, one component of self-confidence, called `solver-quality', is discussed in the context of Markov decision processes for autonomous systems. Markov decision processes underlie much of the theory of reinforcement learning, and are commonly used for planning and decision making under uncertainty in robotics and autonomous systems. A `solver quality' metric is formally defined in the context of decision making algorithms based on Markov decision processes. A method for assessing solver quality is then derived, drawing inspiration from empirical hardness models. Finally, numerical experiments for an unmanned autonomous vehicle navigation problem under different solver, parameter, and environment conditions indicate that the self-confidence metric exhibits the desired properties. Discussion of results, and avenues for future investigation are included.
Adversarial Text Generation Without Reinforcement Learning
Venue: arXiv cs.CL 2019
Year: 2019

Generative Adversarial Networks (GANs) have experienced a recent surge in popularity, performing competitively in a variety of tasks, especially in computer vision. However, GAN training has shown limited success in natural language processing. This is largely because sequences of text are discrete, and thus gradients cannot propagate from the discriminator to the generator. Recent solutions use reinforcement learning to propagate approximate gradients to the generator, but this is inefficient to train. We propose to utilize an autoencoder to learn a low-dimensional representation of sentences. A GAN is then trained to generate its own vectors in this space, which decode to realistic utterances. We report both random and interpolated samples from the generator. Visualization of sentence vectors indicate our model correctly learns the latent space of the autoencoder. Both human ratings and BLEU scores show that our model generates realistic text against competitive baselines.
Deep Transfer Reinforcement Learning for Text Summarization
Venue: arXiv cs.LG 2019
Year: 2019

Deep neural networks are data hungry models and thus face difficulties when attempting to train on small text datasets. Transfer learning is a potential solution but their effectiveness in the text domain is not as explored as in areas such as image analysis. In this paper, we study the problem of transfer learning for text summarization and discuss why existing state-of-the-art models fail to generalize well on other (unseen) datasets. We propose a reinforcement learning framework based on a self-critic policy gradient approach which achieves good generalization and state-of-the-art results on a variety of datasets. Through an extensive set of experiments, we also show the ability of our proposed framework to fine-tune the text summarization model using only a few training samples. To the best of our knowledge, this is the first work that studies transfer learning in text summarization and provides a generic solution that works well on unseen data.
Column generation based math-heuristic for classification trees
Venue: arXiv cs.LG 2019
Year: 2019

This paper explores the use of Column Generation (CG) techniques in constructing univariate binary decision trees for classification tasks. We propose a novel Integer Linear Programming (ILP) formulation, based on root-to-leaf paths in decision trees. The model is solved via a Column Generation based heuristic. To speed up the heuristic, we use a restricted instance data by considering a subset of decision splits, sampled from the solutions of the well-known CART algorithm. Extensive numerical experiments show that our approach is competitive with the state-of-the-art ILP-based algorithms. In particular, the proposed approach is capable of handling big data sets with tens of thousands of data rows. Moreover, for large data sets, it finds solutions competitive to CART.
Optimally rotated coordinate systems for adaptive least-squares regression on sparse grids
Venue: arXiv cs.LG 2019
Year: 2019

For low-dimensional data sets with a large amount of data points, standard kernel methods are usually not feasible for regression anymore. Besides simple linear models or involved heuristic deep learning models, grid-based discretizations of larger (kernel) model classes lead to algorithms, which naturally scale linearly in the amount of data points. For moderate-dimensional or high-dimensional regression tasks, these grid-based discretizations suffer from the curse of dimensionality. Here, sparse grid methods have proven to circumvent this problem to a large extent. In this context, space- and dimension-adaptive sparse grids, which can detect and exploit a given low effective dimensionality of nominally high-dimensional data, are particularly successful. They nevertheless rely on an axis-aligned structure of the solution and exhibit issues for data with predominantly skewed and rotated coordinates. In this paper we propose a preprocessing approach for these adaptive sparse grid algorithms that determines an optimized, problem-dependent coordinate system and, thus, reduces the effective dimensionality of a given data set in the ANOVA sense. We provide numerical examples on synthetic data as well as real-world data to show how an adaptive sparse grid least squares algorithm benefits from our preprocessing method.
Maximizing Monotone DR-submodular Continuous Functions by Derivative-free Optimization
Venue: arXiv cs.LG 2019
Year: 2019

In this paper, we study the problem of monotone (weakly) DR-submodular continuous maximization. While previous methods require the gradient information of the objective function, we propose a derivative-free algorithm LDGM for the first time. We define $β$ and $α$ to characterize how close a function is to continuous DR-submodulr and submodular, respectively. Under a convex polytope constraint, we prove that LDGM can achieve a $(1-e^{-β}-ε)$-approximation guarantee after $O(1/ε)$ iterations, which is the same as the best previous gradient-based algorithm. Moreover, in some special cases, a variant of LDGM can achieve a $((α/2)(1-e^{-α})-ε)$-approximation guarantee for (weakly) submodular functions. We also compare LDGM with the gradient-based algorithm Frank-Wolfe under noise, and show that LDGM can be more robust. Empirical results on budget allocation verify the effectiveness of LDGM.
UnrealROX: An eXtremely Photorealistic Virtual Reality Environment for Robotics Simulations and Synthetic Data Generation
Venue: arXiv cs.RO 2019
Year: 2019

Data-driven algorithms have surpassed traditional techniques in almost every aspect in robotic vision problems. Such algorithms need vast amounts of quality data to be able to work properly after their training process. Gathering and annotating that sheer amount of data in the real world is a time-consuming and error-prone task. Those problems limit scale and quality. Synthetic data generation has become increasingly popular since it is faster to generate and automatic to annotate. However, most of the current datasets and environments lack realism, interactions, and details from the real world. UnrealROX is an environment built over Unreal Engine 4 which aims to reduce that reality gap by leveraging hyperrealistic indoor scenes that are explored by robot agents which also interact with objects in a visually realistic manner in that simulated world. Photorealistic scenes and robots are rendered by Unreal Engine into a virtual reality headset which captures gaze so that a human operator can move the robot and use controllers for the robotic hands; scene information is dumped on a per-frame basis so that it can be reproduced offline to generate raw data and ground truth annotations. This virtual reality environment enables robotic vision researchers to generate realistic and visually plausible data with full ground truth for a wide variety of problems such as class and instance semantic segmentation, object detection, depth estimation, visual grasping, and navigation.
List Decoding of Deletions Using Guess & Check Codes
Venue: arXiv cs.IT 2019
Year: 2019

Guess & Check (GC) codes are systematic binary codes that can correct multiple deletions, with high probability. GC codes have logarithmic redundancy in the length of the message $k$, and the encoding and decoding algorithms of these codes are deterministic and run in polynomial time for a constant number of deletions $δ$. The unique decoding properties of GC codes were examined in a previous work by the authors. In this paper, we investigate the list decoding performance of these codes. Namely, we study the average size and the maximum size of the list obtained by a GC decoder for a constant number of deletions $δ$. The theoretical results show that: (i) the average size of the list approaches $1$ as $k$ grows; and (ii) there exists an infinite sequence of GC codes indexed by $k$, whose maximum list size in upper bounded by a constant that is independent of $k$. We also provide numerical simulations on the list decoding performance of GC codes for multiple values of $k$ and $δ$.
Functionality-Oriented Convolutional Filter Pruning
Venue: arXiv cs.LG 2019
Year: 2019

The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although extremely effective, most works are based only on quantitative characteristics of the convolutional filters, and highly overlook the qualitative interpretation of individual filter's specific functionality. In this work, we interpreted the functionality and redundancy of the convolutional filters from different perspectives, and proposed a functionality-oriented filter pruning method. With extensive experiment results, we proved the convolutional filters' qualitative significance regardless of magnitude, demonstrated significant neural network redundancy due to repetitive filter functions, and analyzed the filter functionality defection under inappropriate retraining process. Such an interpretable pruning approach not only offers outstanding computation cost optimization over previous filter pruning methods, but also interprets filter pruning process.
Learning in Non-convex Games with an Optimization Oracle
Venue: arXiv cs.LG 2019
Year: 2019

We consider online learning in an adversarial, non-convex setting under the assumption that the learner has an access to an offline optimization oracle. In the general setting of prediction with expert advice, Hazan et al. (2016) established that in the optimization-oracle model, online learning requires exponentially more computation than statistical learning. In this paper we show that by slightly strengthening the oracle model, the online and the statistical learning models become computationally equivalent. Our result holds for any Lipschitz and bounded (but not necessarily convex) function. As an application we demonstrate how the offline oracle enables efficient computation of an equilibrium in non-convex games, that include GAN (generative adversarial networks) as a special case.
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations
Venue: arXiv cs.NE 2019
Year: 2019

Temporal models based on recurrent neural networks have proven to be quite powerful in a wide variety of applications. However, training these models often relies on back-propagation through time, which entails unfolding the network over many time steps, making the process of conducting credit assignment considerably more challenging. Furthermore, the nature of back-propagation itself does not permit the use of non-differentiable activation functions and is inherently sequential, making parallelization of the underlying training process difficult. Here, we propose the Parallel Temporal Neural Coding Network (P-TNCN), a biologically inspired model trained by the learning algorithm we call Local Representation Alignment. It aims to resolve the difficulties and problems that plague recurrent networks trained by back-propagation through time. The architecture requires neither unrolling in time nor the derivatives of its internal activation functions. We compare our model and learning procedure to other back-propagation through time alternatives (which also tend to be computationally expensive), including real-time recurrent learning, echo state networks, and unbiased online recurrent optimization. We show that it outperforms these on sequence modeling benchmarks such as Bouncing MNIST, a new benchmark we denote as Bouncing NotMNIST, and Penn Treebank. Notably, our approach can in some instances outperform full back-propagation through time as well as variants such as sparse attentive back-tracking. Significantly, the hidden unit correction phase of P-TNCN allows it to adapt to new datasets even if its synaptic weights are held fixed (zero-shot adaptation) and facilitates retention of prior generative knowledge when faced with a task sequence. We present results that show the P-TNCN's ability to conduct zero-shot adaptation and online continual sequence modeling.
Fast and Longest Rollercoasters
Venue: arXiv cs.DS 2019
Year: 2019

For $k\geq 3$, a k-rollercoaster is a sequence of numbers whose every maximal contiguous subsequence, that is increasing or decreasing, has length at least $k$; $3$-rollercoasters are called simply rollercoasters. Given a sequence of distinct numbers, we are interested in computing its maximum-length (not necessarily contiguous) subsequence that is a $k$-rollercoaster. Biedl et al. [ICALP 2018] have shown that each sequence of $n$ distinct real numbers contains a rollercoaster of length at least $\lceil n/2\rceil$ for $n>7$, and that a longest rollercoaster contained in such a sequence can be computed in $O(n\log n)$-time. They have also shown that every sequence of $n\geq (k-1)^2+1$ distinct real numbers contains a $k$-rollercoaster of length at least $\frac{n}{2(k-1)}-\frac{3k}{2}$, and gave an $O(nk\log n)$-time algorithm computing a longest $k$-rollercoaster in a sequence of length $n$. In this paper, we give an $O(nk^2)$-time algorithm computing the length of a longest $k$-rollercoaster contained in a sequence of $n$ distinct real numbers; hence, for constant $k$, our algorithm computes the length of a longest $k$-rollercoaster in optimal linear time. The algorithm can be easily adapted to output the respective $k$-rollercoaster. In particular, this improves the results of Biedl et al. [ICALP 2018], by showing that a longest rollercoaster can be computed in optimal linear time. We also present an algorithm computing the length of a longest $k$-rollercoaster in $O(n \log^2 n)$-time, that is, subquadratic even for large values of $k\leq n$. Again, the rollercoaster can be easily retrieved. Finally, we show an $Ω(n \log k)$ lower bound for the number of comparisons in any comparison-based algorithm computing the length of a longest $k$-rollercoaster.