OpenRaiser/Intern-Atlas huggingface dataset:
Value Propagation Networks
Venue: arXiv cs.AI 2019
Year: 2019

We present Value Propagation (VProp), a set of parameter-efficient differentiable planning modules built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments. We show that the modules enable learning to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems. We evaluate on static and dynamic configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes, and on a StarCraft navigation scenario, with more complex dynamics, and pixels as input.
A Decentralized Optimal Feedback Flow Control Approach for Transport Networks
Venue: arXiv cs.SY 2019
Year: 2019

Finite-time optimal feedback control for flow networks under information constraints is studied. By utilizing the framework of multi-parametric linear programming, it is demonstrated that when cost/constraints can be modeled or approximated by piecewise-affine functions, the optimal control has a closed-form state-feedback realization. The optimal feedback control law, however, has a centralized structure and requires instantaneous access to the state of the entire network that may lead to prohibitive communication requirements in large-scale complex networks. We subsequently examine the design of a decentralized optimal feedback controller with a one-hop information structure, wherein the optimum outflow rate from each segment of the network depends only on the state of that segment and the state of the segments immediately downstream. The decentralization is based on the relaxation of constraints that depend on state variables that are unavailable according to the information structure. The resulting decentralized control scheme has a simple closed-form representation and is scalable to arbitrary large networks; moreover, we demonstrate that, with respect to certain meaningful performance indexes, the performance loss due to decentralization is zero; namely, the centralized optimal controller has a decentralized realization with a one-hop information structure and is obtained at no computational/communication cost.
Improved Mixed-Example Data Augmentation
Venue: arXiv cs.CV 2019
Year: 2019

In order to reduce overfitting, neural networks are typically trained with data augmentation, the practice of artificially generating additional training data via label-preserving transformations of existing training examples. While these types of transformations make intuitive sense, recent work has demonstrated that even non-label-preserving data augmentation can be surprisingly effective, examining this type of data augmentation through linear combinations of pairs of examples. Despite their effectiveness, little is known about why such methods work. In this work, we aim to explore a new, more generalized form of this type of data augmentation in order to determine whether such linearity is necessary. By considering this broader scope of "mixed-example data augmentation", we find a much larger space of practical augmentation techniques, including methods that improve upon previous state-of-the-art. This generalization has benefits beyond the promise of improved performance, revealing a number of types of mixed-example data augmentation that are radically different from those considered in prior work, which provides evidence that current theories for the effectiveness of such methods are incomplete and suggests that any such theory must explain a much broader phenomenon. Code is available at https://github.com/ceciliaresearch/MixedExample.
Fairness and Sum-Rate Maximization via Joint Channel and Power Allocation in Uplink SCMA Networks
Venue: arXiv cs.IT 2019
Year: 2019

In this work, we consider a sparse code multiple access uplink system, where $J$ users simultaneously transmit data over $K$ subcarriers, such that $J > K$, with a constraint on the power transmitted by each user. To jointly optimize the subcarrier assignment and the transmitted power per subcarrier, two new iterative algorithms are proposed, the first one aims to maximize the sum-rate (Max-SR) of the network, while the second aims to maximize the fairness (Max-Min). In both cases, the optimization problem is of the mixed-integer nonlinear programming (MINLP) type, with non-convex objective functions, which are generally not tractable. We prove that both joint allocation problems are NP-hard. To address these issues, we employ a variant of the block successive upper-bound minimization (BSUM) \cite{razaviyayn.2013} framework, obtaining polynomial-time approximation algorithms to the original problem. Moreover, we evaluate the algorithms' robustness against outdated channel state information (CSI), present an analysis of the convergence of the algorithms, and a comparison of the sum-rate and Jain's fairness index of the novel algorithms with three other algorithms proposed in the literature. The Max-SR algorithm outperforms the others in the sum-rate sense, while the Max-Min outperforms them in the fairness sense.
Approximate LTL model checking
Venue: arXiv cs.LO 2019
Year: 2019

The state explosion problem and the exponentially computational complexity restrict the further applications of LTL model checking. To this end, this study tries to seek an acceptable approximate solution for LTL model checking by introducing the Machine Learning (ML) technique, and a method for predicting results of LTL model checking via the Boosted Tree (BT) algorithm is proposed in this paper. First, for a number of Kripke structures and LTL formulas, a data set A containing their model checking results is obtained, using the existing LTL model checking algorithm. Second, the LTL model checking problem can be induced to a binary classification problem of machine learning. In other words, some records in A form a training set for the BT algorithm. On the basis of it, a ML model M is obtained to predict the results of LTL model checking. As a result, an approximate LTL model checking technique occurs. The experiments show that the new method has the average accuracy of 98.0%, and its average efficiency is 9.4 million times higher than that of the representative model checking method, if the length of each of LTL formulas equals to 500.
Multi-level 3D CNN for Learning Multi-scale Spatial Features
Venue: arXiv cs.CV 2019
Year: 2019

3D object recognition accuracy can be improved by learning the multi-scale spatial features from 3D spatial geometric representations of objects such as point clouds, 3D models, surfaces, and RGB-D data. Current deep learning approaches learn such features either using structured data representations (voxel grids and octrees) or from unstructured representations (graphs and point clouds). Learning features from such structured representations is limited by the restriction on resolution and tree depth while unstructured representations creates a challenge due to non-uniformity among data samples. In this paper, we propose an end-to-end multi-level learning approach on a multi-level voxel grid to overcome these drawbacks. To demonstrate the utility of the proposed multi-level learning, we use a multi-level voxel representation of 3D objects to perform object recognition. The multi-level voxel representation consists of a coarse voxel grid that contains volumetric information of the 3D object. In addition, each voxel in the coarse grid that contains a portion of the object boundary is subdivided into multiple fine-level voxel grids. The performance of our multi-level learning algorithm for object recognition is comparable to dense voxel representations while using significantly lower memory.
A Robust Iterative Scheme for Symmetric Indefinite Systems
Venue: arXiv cs.NA 2019
Year: 2019

We propose a two-level nested preconditioned iterative scheme for solving sparse linear systems of equations in which the coefficient matrix is symmetric and indefinite with relatively small number of negative eigenvalues. The proposed scheme consists of an outer Minimum Residual (MINRES) iteration, preconditioned by an inner Conjugate Gradient (CG) iteration in which CG can be further preconditioned. The robustness of the proposed scheme is illustrated by solving indefinite linear systems that arise in the solution of quadratic eigenvalue problems in the context of model reduction methods for finite element models of disk brakes as well as on other problems that arise in a variety of applications.
A faster hafnian formula for complex matrices and its benchmarking on a supercomputer
Venue: arXiv cs.DS 2019
Year: 2019

We introduce new and simple algorithms for the calculation of the number of perfect matchings of complex weighted, undirected graphs with and without loops. Our compact formulas for the hafnian and loop hafnian of $n \times n $ complex matrices run in $O(n^3 2^{n/2})$ time, are embarrassingly parallelizable and, to the best of our knowledge, are the fastest exact algorithms to compute these quantities. Despite our highly optimized algorithm, numerical benchmarks on the Titan supercomputer with matrices up to size $56 \times 56$ indicate that one would require the 288000 CPUs of this machine for about a month and a half to compute the hafnian of a $100 \times 100$ matrix.
k-Space Deep Learning for Reference-free EPI Ghost Correction
Venue: arXiv cs.CV 2019
Year: 2019

Nyquist ghost artifacts in EPI are originated from phase mismatch between the even and odd echoes. However, conventional correction methods using reference scans often produce erroneous results especially in high-field MRI due to the non-linear and time-varying local magnetic field changes. Recently, it was shown that the problem of ghost correction can be reformulated as k-space interpolation problem that can be solved using structured low-rank Hankel matrix approaches. Another recent work showed that data driven Hankel matrix decomposition can be reformulated to exhibit similar structures as deep convolutional neural network. By synergistically combining these findings, we propose a k-space deep learning approach that immediately corrects the phase mismatch without a reference scan in both accelerated and non-accelerated EPI acquisitions. To take advantage of the even and odd-phase directional redundancy, the k-space data is divided into two channels configured with even and odd phase encodings. The redundancies between coils are also exploited by stacking the multi-coil k-space data into additional input channels. Then, our k-space ghost correction network is trained to learn the interpolation kernel to estimate the missing virtual k-space data. For the accelerated EPI data, the same neural network is trained to directly estimate the interpolation kernels for missing k-space data from both ghost and subsampling. Reconstruction results using 3T and 7T in-vivo data showed that the proposed method outperformed the image quality compared to the existing methods, and the computing time is much faster.The proposed k-space deep learning for EPI ghost correction is highly robust and fast, and can be combined with acceleration, so that it can be used as a promising correction tool for high-field MRI without changing the current acquisition protocol.
On Performance of Peer Review for Academic Journals: Analysis Based on Distributed Parallel System
Venue: arXiv cs.DL 2019
Year: 2019

A simulation model based on parallel systems is established, aiming to explore the relation between the number of submissions and the overall standard of academic journals within a similar discipline under peer review. The model can effectively simulate the submission, review, and acceptance behaviors of academic journals in a distributed manner. According to the simulation experiments, it could possibly happen that the overall standard of academic journals deteriorates due to excessive submissions.
Smart Contracts for Machine-to-Machine Communication: Possibilities and Limitations
Venue: arXiv cs.CY 2019
Year: 2019

Blockchain technologies, such as smart contracts, present a unique interface for machine-to-machine communication that provides a secure, append-only record that can be shared without trust and without a central administrator. We study the possibilities and limitations of using smart contracts for machine-to-machine communication by designing, implementing, and evaluating AGasP, an application for automated gasoline purchases. We find that using smart contracts allows us to directly address the challenges of transparency, longevity, and trust in IoT applications. However, real-world applications using smart contracts must address their important trade-offs, such as performance, privacy, and the challenge of ensuring they are written correctly.
A Hierarchical Attention Model for Social Contextual Image Recommendation
Venue: arXiv cs.SI 2019
Year: 2019

Image based social networks are among the most popular social networking services in recent years. With tremendous images uploaded everyday, understanding users' preferences on user-generated images and making recommendations have become an urgent need. In fact, many hybrid models have been proposed to fuse various kinds of side information~(e.g., image visual representation, social network) and user-item historical behavior for enhancing recommendation performance. However, due to the unique characteristics of the user generated images in social image platforms, the previous studies failed to capture the complex aspects that influence users' preferences in a unified framework. Moreover, most of these hybrid models relied on predefined weights in combining different kinds of information, which usually resulted in sub-optimal recommendation performance. To this end, in this paper, we develop a hierarchical attention model for social contextual image recommendation. In addition to basic latent user interest modeling in the popular matrix factorization based recommendation, we identify three key aspects (i.e., upload history, social influence, and owner admiration) that affect each user's latent preferences, where each aspect summarizes a contextual factor from the complex relationships between users and images. After that, we design a hierarchical attention network that naturally mirrors the hierarchical relationship (elements in each aspects level, and the aspect level) of users' latent interests with the identified key aspects. Specifically, by taking embeddings from state-of-the-art deep learning models that are tailored for each kind of data, the hierarchical attention network could learn to attend differently to more or less content. Finally, extensive experimental results on real-world datasets clearly show the superiority of our proposed model.
Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization
Venue: arXiv cs.LG 2019
Year: 2019

Despite the success of single-agent reinforcement learning, multi-agent reinforcement learning (MARL) remains challenging due to complex interactions between agents. Motivated by decentralized applications such as sensor networks, swarm robotics, and power grids, we study policy evaluation in MARL, where agents with jointly observed state-action pairs and private local rewards collaborate to learn the value of a given policy. In this paper, we propose a double averaging scheme, where each agent iteratively performs averaging over both space and time to incorporate neighboring gradient information and local reward information, respectively. We prove that the proposed algorithm converges to the optimal solution at a global geometric rate. In particular, such an algorithm is built upon a primal-dual reformulation of the mean squared projected Bellman error minimization problem, which gives rise to a decentralized convex-concave saddle-point problem. To the best of our knowledge, the proposed double averaging primal-dual optimization algorithm is the first to achieve fast finite-time convergence on decentralized convex-concave saddle-point problems.
NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Scene text recognition has attracted a great many researches due to its importance to various applications. Existing methods mainly adopt recurrence or convolution based networks. Though have obtained good performance, these methods still suffer from two limitations: slow training speed due to the internal recurrence of RNNs, and high complexity due to stacked convolutional layers for long-term feature extraction. This paper, for the first time, proposes a no-recurrence sequence-to-sequence text recognizer, named NRTR, that dispenses with recurrences and convolutions entirely. NRTR follows the encoder-decoder paradigm, where the encoder uses stacked self-attention to extract image features, and the decoder applies stacked self-attention to recognize texts based on encoder output. NRTR relies solely on self-attention mechanism thus could be trained with more parallelization and less complexity. Considering scene image has large variation in text and background, we further design a modality-transform block to effectively transform 2D input images to 1D sequences, combined with the encoder to extract more discriminative features. NRTR achieves state-of-the-art or highly competitive performance on both regular and irregular benchmarks, while requires only a small fraction of training time compared to the best model from the literature (at least 8 times faster).
REORDER: Securing Dynamic-Priority Real-Time Systems Using Schedule Obfuscation
Venue: arXiv cs.CR 2019
Year: 2019

The deterministic (timing) behavior of real-time systems (RTS) can be used by adversaries - say, to launch side channel attacks or even destabilize the system by denying access to critical resources. We propose a protocol (named REORDER) to obfuscate this predictable timing behavior of RTS, especially ones designed using dynamic-priority scheduling algorithms (e.g., EDF). We also present a metric (named "schedule entropy") that measures the levels of obfuscation introduced into a given real-time system. The REORDER protocol was integrated into the standard Linux real-time scheduler and evaluated on a realistic embedded platform (Raspberry Pi) running the MiBench automotive benchmark workloads. We also demonstrate how designers of RTS can increase the security of their systems and also quantitatively measure the impact (both in terms of security and performance) of using this protocol.
Attention Based Fully Convolutional Network for Speech Emotion Recognition
Venue: arXiv cs.SD 2019
Year: 2019

Speech emotion recognition is a challenging task for three main reasons: 1) human emotion is abstract, which means it is hard to distinguish; 2) in general, human emotion can only be detected in some specific moments during a long utterance; 3) speech data with emotional labeling is usually limited. In this paper, we present a novel attention based fully convolutional network for speech emotion recognition. We employ fully convolutional network as it is able to handle variable-length speech, free of the demand of segmentation to keep critical information not lost. The proposed attention mechanism can make our model be aware of which time-frequency region of speech spectrogram is more emotion-relevant. Considering limited data, the transfer learning is also adapted to improve the accuracy. Especially, it's interesting to observe obvious improvement obtained with natural scene image based pre-trained model. Validated on the publicly available IEMOCAP corpus, the proposed model outperformed the state-of-the-art methods with a weighted accuracy of 70.4% and an unweighted accuracy of 63.9% respectively.
Recurrent Convolutional Fusion for RGB-D Object Recognition
Venue: arXiv cs.CV 2019
Year: 2019

Providing machines with the ability to recognize objects like humans has always been one of the primary goals of machine vision. The introduction of RGB-D cameras has paved the way for a significant leap forward in this direction thanks to the rich information provided by these sensors. However, the machine vision community still lacks an effective method to synergically use the RGB and depth data to improve object recognition. In order to take a step in this direction, we introduce a novel end-to-end architecture for RGB-D object recognition called recurrent convolutional fusion (RCFusion). Our method generates compact and highly discriminative multi-modal features by combining complementary RGB and depth information representing different levels of abstraction. Extensive experiments on two popular datasets, RGB-D Object Dataset and JHUIT-50, show that RCFusion significantly outperforms state-of-the-art approaches in both the object categorization and instance recognition tasks.
A Memristor based Unsupervised Neuromorphic System Towards Fast and Energy-Efficient GAN
Venue: arXiv cs.DC 2019
Year: 2019

Deep Learning has gained immense success in pushing today's artificial intelligence forward. To solve the challenge of limited labeled data in the supervised learning world, unsupervised learning has been proposed years ago while low accuracy hinters its realistic applications. Generative adversarial network (GAN) emerges as an unsupervised learning approach with promising accuracy and are under extensively study. However, the execution of GAN is extremely memory and computation intensive and results in ultra-low speed and high-power consumption. In this work, we proposed a holistic solution for fast and energy-efficient GAN computation through a memristor-based neuromorphic system. First, we exploited a hardware and software co-design approach to map the computation blocks in GAN efficiently. We also proposed an efficient data flow for optimal parallelism training and testing, depending on the computation correlations between different computing blocks. To compute the unique and complex loss of GAN, we developed a diff-block with optimized accuracy and performance. The experiment results on big data show that our design achieves 2.8x speedup and 6.1x energy-saving compared with the traditional GPU accelerator, as well as 5.5x speedup and 1.4x energy-saving compared with the previous FPGA-based accelerator.
Focal Visual-Text Attention for Visual Question Answering
Venue: arXiv cs.CV 2019
Year: 2019

Recent insights on language and vision with neural networks have been successfully applied to simple single-image visual question answering. However, to tackle real-life question answering problems on multimedia collections such as personal photos, we have to look at whole collections with sequences of photos or videos. When answering questions from a large collection, a natural problem is to identify snippets to support the answer. In this paper, we describe a novel neural network called Focal Visual-Text Attention network (FVTA) for collective reasoning in visual question answering, where both visual and text sequence information such as images and text metadata are presented. FVTA introduces an end-to-end approach that makes use of a hierarchical process to dynamically determine what media and what time to focus on in the sequential data to answer the question. FVTA can not only answer the questions well but also provides the justifications which the system results are based upon to get the answers. FVTA achieves state-of-the-art performance on the MemexQA dataset and competitive results on the MovieQA dataset.