OpenRaiser/Intern-Atlas huggingface dataset:
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
Venue: arXiv cs.CV 2017
Year: 2017

In this paper, we propose an Attentional Generative Adversarial Network (AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained text-to-image generation. With a novel attentional generative network, the AttnGAN can synthesize fine-grained details at different subregions of the image by paying attentions to the relevant words in the natural language description. In addition, a deep attentional multimodal similarity model is proposed to compute a fine-grained image-text matching loss for training the generator. The proposed AttnGAN significantly outperforms the previous state of the art, boosting the best reported inception score by 14.14% on the CUB dataset and 170.25% on the more challenging COCO dataset. A detailed analysis is also performed by visualizing the attention layers of the AttnGAN. It for the first time shows that the layered attentional GAN is able to automatically select the condition at the word level for generating different parts of the image.
A Recursive Bayesian Approach To Describe Retinal Vasculature Geometry
Venue: arXiv cs.CV 2017
Year: 2017

Demographic studies suggest that changes in the retinal vasculature geometry, especially in vessel width, are associated with the incidence or progression of eye-related or systemic diseases. To date, the main information source for width estimation from fundus images has been the intensity profile between vessel edges. However, there are many factors affecting the intensity profile: pathologies, the central light reflex and local illumination levels, to name a few. In this study, we introduce three information sources for width estimation. These are the probability profiles of vessel interior, centreline and edge locations generated by a deep network. The probability profiles provide direct access to vessel geometry and are used in the likelihood calculation for a Bayesian method, particle filtering. We also introduce a geometric model which can handle non-ideal conditions of the probability profiles. Our experiments conducted on the REVIEW dataset yielded consistent estimates of vessel width, even in cases when one of the vessel edges is difficult to identify. Moreover, our results suggest that the method is better than human observers at locating edges of low contrast vessels.
Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
Venue: arXiv cs.AI 2017
Year: 2017

We introduce physics informed neural networks -- neural networks that are trained to solve supervised learning tasks while respecting any given law of physics described by general nonlinear partial differential equations. In this second part of our two-part treatise, we focus on the problem of data-driven discovery of partial differential equations. Depending on whether the available data is scattered in space-time or arranged in fixed temporal snapshots, we introduce two main classes of algorithms, namely continuous time and discrete time models. The effectiveness of our approach is demonstrated using a wide range of benchmark problems in mathematical physics, including conservation laws, incompressible fluid flow, and the propagation of nonlinear shallow-water waves.
A Survey of Learning Management Systems and Synchronous Distance Education Tools
Venue: arXiv cs.CY 2017
Year: 2017

Although compelling assessments have been quite frequently examined in recent years, more studies are required to yield a better understanding of several Distance Learning (DL) methods where Learning Management Systems (LMSs) significantly affect student learning process. Most studies in this area do not consider the effect of varying web-facilitated DL application tools. To address these drawbacks, the objective of our study is to compare two LMSs and four synchronous distance education tools (SDET). The comparisons confirm the superiority of Moodle Integrated Synchrotrons Teaching Conferencing (MIST/C), which seems to be the most practical, convenient and modest distance education tool offered in the market today because it is open source and has a second mirrored whiteboard for simulteaching that is not available with any other system.
PSIque: Next Sequence Prediction of Satellite Images using a Convolutional Sequence-to-Sequence Network
Venue: arXiv cs.CV 2017
Year: 2017

Predicting unseen weather phenomena is an important issue for disaster management. In this paper, we suggest a model for a convolutional sequence-to-sequence autoencoder for predicting undiscovered weather situations from previous satellite images. We also propose a symmetric skip connection between encoder and decoder modules to produce more comprehensive image predictions. To examine our model performance, we conducted experiments for each suggested model to predict future satellite images from historical satellite images. A specific combination of skip connection and sequence-to-sequence autoencoder was able to generate closest prediction from the ground truth image.
Chance-Constrained Day-Ahead Hourly Scheduling in Distribution System Operation
Venue: arXiv cs.SY 2017
Year: 2017

This paper aims to propose a two-step approach for day-ahead hourly scheduling in a distribution system operation, which contains two operation costs, the operation cost at substation level and feeder level. In the first step, the objective is to minimize the electric power purchase from the day-ahead market with the stochastic optimization. The historical data of day-ahead hourly electric power consumption is used to provide the forecast results with the forecasting error, which is presented by a chance constraint and formulated into a deterministic form by Gaussian mixture model (GMM). In the second step, the objective is to minimize the system loss. Considering the nonconvexity of the three-phase balanced AC optimal power flow problem in distribution systems, the second-order cone program (SOCP) is used to relax the problem. Then, a distributed optimization approach is built based on the alternating direction method of multiplier (ADMM). The results shows that the validity and effectiveness method.
BLADE: Filter Learning for General Purpose Computational Photography
Venue: arXiv cs.CV 2017
Year: 2017

The Rapid and Accurate Image Super Resolution (RAISR) method of Romano, Isidoro, and Milanfar is a computationally efficient image upscaling method using a trained set of filters. We describe a generalization of RAISR, which we name Best Linear Adaptive Enhancement (BLADE). This approach is a trainable edge-adaptive filtering framework that is general, simple, computationally efficient, and useful for a wide range of problems in computational photography. We show applications to operations which may appear in a camera pipeline including denoising, demosaicing, and stylization.
Transfer Learning with Binary Neural Networks
Venue: arXiv cs.NE 2017
Year: 2017

Previous work has shown that it is possible to train deep neural networks with low precision weights and activations. In the extreme case it is even possible to constrain the network to binary values. The costly floating point multiplications are then reduced to fast logical operations. High end smart phones such as Google's Pixel 2 and Apple's iPhone X are already equipped with specialised hardware for image processing and it is very likely that other future consumer hardware will also have dedicated accelerators for deep neural networks. Binary neural networks are attractive in this case because the logical operations are very fast and efficient when implemented in hardware. We propose a transfer learning based architecture where we first train a binary network on Imagenet and then retrain part of the network for different tasks while keeping most of the network fixed. The fixed binary part could be implemented in a hardware accelerator while the last layers of the network are evaluated in software. We show that a single binary neural network trained on the Imagenet dataset can indeed be used as a feature extractor for other datasets.
Partial Consensus and Conservative Fusion of Gaussian Mixtures for Distributed PHD Fusion
Venue: arXiv cs.SY 2017
Year: 2017

We propose a novel consensus notion, called "partial consensus", for distributed GM-PHD (Gaussian mixture probability hypothesis density) fusion based on a peer-to-peer (P2P) sensor network, in which only highly-weighted posterior Gaussian components (GCs) are disseminated in the P2P communication for fusion while the insignificant GCs are not involved. The partial consensus does not only enjoy high efficiency in both network communication and local fusion computation, but also significantly reduces the affect of potential false data (clutter) to the filter, leading to increased signal-to-noise ratio at local sensors. Two "conservative" mixture reduction schemes are advocated for fusing the shared GCs in a fully distributed manner. One is given by pairwise averaging GCs between sensors based on Hungarian assignment and the other is merging close GCs based a new GM merging scheme. The proposed approaches have a close connection to the conservative fusion approaches known as covariance union and arithmetic mean density. In parallel, average consensus is sought on the cardinality distribution (namely the GM weight sum) among sensors. Simulations for tracking either a single target or multiple targets that simultaneously appear are presented based on a sensor network where each sensor operates a GM-PHD filter, in order to compare our approaches with the benchmark generalized covariance intersection approach. The results demonstrate that the partial, arithmetic average, consensus outperforms the complete, geometric average, consensus.
Efficient exploration with Double Uncertain Value Networks
Venue: arXiv cs.LG 2017
Year: 2017

This paper studies directed exploration for reinforcement learning agents by tracking uncertainty about the value of each available action. We identify two sources of uncertainty that are relevant for exploration. The first originates from limited data (parametric uncertainty), while the second originates from the distribution of the returns (return uncertainty). We identify methods to learn these distributions with deep neural networks, where we estimate parametric uncertainty with Bayesian drop-out, while return uncertainty is propagated through the Bellman equation as a Gaussian distribution. Then, we identify that both can be jointly estimated in one network, which we call the Double Uncertain Value Network. The policy is directly derived from the learned distributions based on Thompson sampling. Experimental results show that both types of uncertainty may vastly improve learning in domains with a strong exploration challenge.
Downlink Precoding with Mixed Statistical and Imperfect Instantaneous CSI for Massive MIMO Systems
Venue: arXiv cs.IT 2017
Year: 2017

In this paper, the feasibility of a new downlink transmission mode in massive multi-input multi-output (MIMO) systems is investigated with two types of users, i.e., the users with only statistical channel state information (CSI) and the users with imperfect instantaneous CSI. The problem of downlink precoding design with mixed utilization of statistical and imperfect instantaneous CSI is addressed. We first theoretically analyze the impact of the mutual interference between the two types of users on their achievable rate. Then, considering the mutual interference suppression, we propose an extended zero-forcing (eZF) and an extended maximum ratio transmission (eMRT) precoding methods to minimize the total transmit power of base station and to maximize the received signal power of users, respectively. Thanks to the exploitation of statistical CSI, pilot-based channel estimation is avoided enabling more active users, higher system sum rate and shorter transmission delay. Finally, simulations are performed to validate the accuracy of the theoretical analysis and the advantages of the proposed precoding methods.
Curriculum Q-Learning for Visual Vocabulary Acquisition
Venue: arXiv cs.CL 2017
Year: 2017

The structure of curriculum plays a vital role in our learning process, both as children and adults. Presenting material in ascending order of difficulty that also exploits prior knowledge can have a significant impact on the rate of learning. However, the notion of difficulty and prior knowledge differs from person to person. Motivated by the need for a personalised curriculum, we present a novel method of curriculum learning for vocabulary words in the form of visual prompts. We employ a reinforcement learning model grounded in pedagogical theories that emulates the actions of a tutor. We simulate three students with different levels of vocabulary knowledge in order to evaluate the how well our model adapts to the environment. The results of the simulation reveal that through interaction, the model is able to identify the areas of weakness, as well as push students to the edge of their ZPD. We hypothesise that these methods can also be effective in training agents to learn language representations in a simulated environment where it has previously been shown that order of words and prior knowledge play an important role in the efficacy of language learning.
Learning Interesting Categorical Attributes for Refined Data Exploration
Venue: arXiv cs.DB 2017
Year: 2017

This work proposes and evaluates a novel approach to determine interesting categorical attributes for lists of entities. Once identified, such categories are of immense value to allow constraining (filtering) a current view of a user to subsets of entities. We show how a classifier is trained that is able to tell whether or not a categorical attribute can act as a constraint, in the sense of human-perceived interestingness. The training data is harnessed from Web tables, treating the presence or absence of a table as an indication that the attribute used as a filter constraint is reasonable or not. For learning the classification model, we review four well-known statistical measures (features) for categorical attributes---entropy, unalikeability, peculiarity, and coverage. We additionally propose three new statistical measures to capture the distribution of data, tailored to our main objective. The learned model is evaluated by relevance assessments obtained through a user study, reflecting the applicability of the approach as a whole and, further, demonstrates the superiority of the proposed diversity measures over existing statistical measures like information entropy.
NPC: Neighbors Progressive Competition Algorithm for Classification of Imbalanced Data Sets
Venue: arXiv cs.LG 2017
Year: 2017

Learning from many real-world datasets is limited by a problem called the class imbalance problem. A dataset is imbalanced when one class (the majority class) has significantly more samples than the other class (the minority class). Such datasets cause typical machine learning algorithms to perform poorly on the classification task. To overcome this issue, this paper proposes a new approach Neighbors Progressive Competition (NPC) for classification of imbalanced datasets. Whilst the proposed algorithm is inspired by weighted k-Nearest Neighbor (k-NN) algorithms, it has major differences from them. Unlike k- NN, NPC does not limit its decision criteria to a preset number of nearest neighbors. In contrast, NPC considers progressively more neighbors of the query sample in its decision making until the sum of grades for one class is much higher than the other classes. Furthermore, NPC uses a novel method for grading the training samples to compensate for the imbalance issue. The grades are calculated using both local and global information. In brief, the contribution of this paper is an entirely new classifier for handling the imbalance issue effectively without any manually-set parameters or any need for expert knowledge. Experimental results compare the proposed approach with five representative algorithms applied to fifteen imbalanced datasets and illustrate this algorithms effectiveness.
Connectivity jamming game for physical layer attack in peer to peer networks
Venue: arXiv cs.GT 2017
Year: 2017

Because of the open access nature of wireless communications, wireless networks can suffer from malicious activity, such as jamming attacks, aimed at undermining the network's ability to sustain communication links and acceptable throughput. One important consideration when designing networks is to appropriately tune the network topology and its connectivity so as to support the communication needs of those participating in the network. This paper examines the problem of interference attacks that are intended to harm connectivity and throughput, and illustrates the method of mapping network performance parameters into the metric of topographic connectivity. Specifically, this paper arrives at anti-jamming strategies aimed at coping with interference attacks through a unified stochastic game. In such a framework, an entity trying to protect a network faces a dilemma: (i) the underlying motivations for the adversary can be quite varied, which depends largely on the network's characteristics such as power and distance; (ii) the metrics for such an attack can be incomparable (e.g., network connectivity and total throughput). To deal with the problem of such incomparable metrics, this paper proposes using the attack's expected duration as a unifying metric to compare distinct attack metrics because a longer-duration of unsuccessful attack assumes a higher cost. Based on this common metric, a mechanism of maxmin selection for an attack prevention strategy is suggested.
Now Playing: Continuous low-power music recognition
Venue: arXiv cs.SD 2017
Year: 2017

Existing music recognition applications require a connection to a server that performs the actual recognition. In this paper we present a low-power music recognizer that runs entirely on a mobile device and automatically recognizes music without user interaction. To reduce battery consumption, a small music detector runs continuously on the mobile device's DSP chip and wakes up the main application processor only when it is confident that music is present. Once woken, the recognizer on the application processor is provided with a few seconds of audio which is fingerprinted and compared to the stored fingerprints in the on-device fingerprint database of tens of thousands of songs. Our presented system, Now Playing, has a daily battery usage of less than 1% on average, respects user privacy by running entirely on-device and can passively recognize a wide range of music.
A Family of Iterative Gauss-Newton Shooting Methods for Nonlinear Optimal Control
Venue: arXiv cs.SY 2017
Year: 2017

This paper introduces a family of iterative algorithms for unconstrained nonlinear optimal control. We generalize the well-known iLQR algorithm to different multiple-shooting variants, combining advantages like straight-forward initialization and a closed-loop forward integration. All algorithms have similar computational complexity, i.e. linear complexity in the time horizon, and can be derived in the same computational framework. We compare the full-step variants of our algorithms and present several simulation examples, including a high-dimensional underactuated robot subject to contact switches. Simulation results show that our multiple-shooting algorithms can achieve faster convergence, better local contraction rates and much shorter runtimes than classical iLQR, which makes them a superior choice for nonlinear model predictive control applications.
Happiness Pursuit: Personality Learning in a Society of Agents
Venue: arXiv cs.MA 2017
Year: 2017

Modeling personality is a challenging problem with applications spanning computer games, virtual assistants, online shopping and education. Many techniques have been tried, ranging from neural networks to computational cognitive architectures. However, most approaches rely on examples with hand-crafted features and scenarios. Here, we approach learning a personality by training agents using a Deep Q-Network (DQN) model on rewards based on psychoanalysis, against hand-coded AI in the game of Pong. As a result, we obtain 4 agents, each with its own personality. Then, we define happiness of an agent, which can be seen as a measure of alignment with agent's objective function, and study it when agents play both against hand-coded AI, and against each other. We find that the agents that achieve higher happiness during testing against hand-coded AI, have lower happiness when competing against each other. This suggests that higher happiness in testing is a sign of overfitting in learning to interact with hand-coded AI, and leads to worse performance against agents with different personalities.
A fast nonconvex Compressed Sensing algorithm for highly low-sampled MR images reconstruction
Venue: arXiv cs.NA 2017
Year: 2017

In this paper we present a fast and efficient method for the reconstruction of Magnetic Resonance Images (MRI) from severely under-sampled data. From the Compressed Sensing theory we have mathematically modeled the problem as a constrained minimization problem with a family of non-convex regularizing objective functions depending on a parameter and a least squares data fit constraint. We propose a fast and efficient algorithm, named Fast NonConvex Reweighting (FNCR) algorithm, based on an iterative scheme where the non-convex problem is approximated by its convex linearization and the penalization parameter is automatically updated. The convex problem is solved by a Forward-Backward procedure, where the Backward step is performed by a Split Bregman strategy. Moreover, we propose a new efficient iterative solver for the arising linear systems. We prove the convergence of the proposed FNCR method. The results on synthetic phantoms and real images show that the algorithm is very well performing and computationally efficient, even when compared to the best performing methods proposed in the literature.
Variational Deep Q Network
Venue: arXiv cs.LG 2017
Year: 2017

We propose a framework that directly tackles the probability distribution of the value function parameters in Deep Q Network (DQN), with powerful variational inference subroutines to approximate the posterior of the parameters. We will establish the equivalence between our proposed surrogate objective and variational inference loss. Our new algorithm achieves efficient exploration and performs well on large scale chain Markov Decision Process (MDP).
A modeling and algorithmic framework for (non)social (co)sparse audio restoration
Venue: arXiv cs.SD 2017
Year: 2017

We propose a unified modeling and algorithmic framework for audio restoration problem. It encompasses analysis sparse priors as well as more classical synthesis sparse priors, and regular sparsity as well as various forms of structured sparsity embodied by shrinkage operators (such as social shrinkage). The versatility of the framework is illustrated on two restoration scenarios: denoising, and declipping. Extensive experimental results on these scenarios highlight both the speedups of 20% or even more offered by the analysis sparse prior, and the substantial declipping quality that is achievable with both the social and the plain flavor. While both flavors overall exhibit similar performance, their detailed comparison displays distinct trends depending whether declipping or denoising is considered.