OpenRaiser/Intern-Atlas huggingface dataset:
Age Progression and Regression with Spatial Attention Modules
Venue: arXiv cs.CV 2019
Year: 2019

Age progression and regression refers to aesthetically render-ing a given face image to present effects of face aging and rejuvenation, respectively. Although numerous studies have been conducted in this topic, there are two major problems: 1) multiple models are usually trained to simulate different age mappings, and 2) the photo-realism of generated face images is heavily influenced by the variation of training images in terms of pose, illumination, and background. To address these issues, in this paper, we propose a framework based on conditional Generative Adversarial Networks (cGANs) to achieve age progression and regression simultaneously. Particularly, since face aging and rejuvenation are largely different in terms of image translation patterns, we model these two processes using two separate generators, each dedicated to one age changing process. In addition, we exploit spatial attention mechanisms to limit image modifications to regions closely related to age changes, so that images with high visual fidelity could be synthesized for in-the-wild cases. Experiments on multiple datasets demonstrate the ability of our model in synthesizing lifelike face images at desired ages with personalized features well preserved, and keeping age-irrelevant regions unchanged.
SNU_IDS at SemEval-2019 Task 3: Addressing Training-Test Class Distribution Mismatch in Conversational Classification
Venue: arXiv cs.CL 2019
Year: 2019

We present several techniques to tackle the mismatch in class distributions between training and test data in the Contextual Emotion Detection task of SemEval 2019, by extending the existing methods for class imbalance problem. Reducing the distance between the distribution of prediction and ground truth, they consistently show positive effects on the performance. Also we propose a novel neural architecture which utilizes representation of overall context as well as of each utterance. The combination of the methods and the models achieved micro F1 score of about 0.766 on the final evaluation.
Camera Obscurer: Generative Art for Design Inspiration
Venue: arXiv cs.MM 2019
Year: 2019

We investigate using generated decorative art as a source of inspiration for design tasks. Using a visual similarity search for image retrieval, the \emph{Camera Obscurer} app enables rapid searching of tens of thousands of generated abstract images of various types. The seed for a visual similarity search is a given image, and the retrieved generated images share some visual similarity with the seed. Implemented in a hand-held device, the app empowers users to use photos of their surroundings to search through the archive of generated images and other image archives. Being abstract in nature, the retrieved images supplement the seed image rather than replace it, providing different visual stimuli including shapes, colours, textures and juxtapositions, in addition to affording their own interpretations. This approach can therefore be used to provide inspiration for a design task, with the abstract images suggesting new ideas that might give direction to a graphic design project. We describe a crowdsourcing experiment with the app to estimate user confidence in retrieved images, and we describe a pilot study where Camera Obscurer provided inspiration for a design task. These experiments have enabled us to describe future improvements, and to begin to understand sources of visual inspiration for design tasks.
Autonomy, Authenticity, Authorship and Intention in computer generated art
Venue: arXiv cs.AI 2019
Year: 2019

This paper examines five key questions surrounding computer generated art. Driven by the recent public auction of a work of `AI Art' we selectively summarise many decades of research and commentary around topics of autonomy, authenticity, authorship and intention in computer generated art, and use this research to answer contemporary questions often asked about art made by computers that concern these topics. We additionally reflect on whether current techniques in deep learning and Generative Adversarial Networks significantly change the answers provided by many decades of prior research.
Synthesizing Chemical Plant Operation Procedures using Knowledge, Dynamic Simulation and Deep Reinforcement Learning
Venue: arXiv cs.AI 2019
Year: 2019

Chemical plants are complex and dynamical systems consisting of many components for manipulation and sensing, whose state transitions depend on various factors such as time, disturbance, and operation procedures. For the purpose of supporting human operators of chemical plants, we are developing an AI system that can semi-automatically synthesize operation procedures for efficient and stable operation. Our system can provide not only appropriate operation procedures but also reasons why the procedures are considered to be valid. This is achieved by integrating automated reasoning and deep reinforcement learning technologies with a chemical plant simulator and external knowledge. Our preliminary experimental results demonstrate that it can synthesize a procedure that achieves a much faster recovery from a malfunction compared to standard PID control.
Proving the NP-completeness of optimal moral graph triangulation
Venue: arXiv cs.CC 2019
Year: 2019

Moral graphs were introduced in the 1980s as an intermediate step when transforming a Bayesian network to a junction tree, on which exact belief propagation can be efficiently done. The moral graph of a Bayesian network can be trivially obtained by connecting non-adjacent parents for each node in the Bayesian network and dropping the direction of each edge. Perhaps because the moralization process looks simple, there has been little attention on the properties of moral graphs and their impact in belief propagation on Bayesian networks. This paper addresses the mistaken claim that it has been previously proved that optimal moral graph triangulation with the constraints of minimum fill-in, treewidth or total states is NP-complete. The problems are in fact NP-complete, but they have not previously been proved. We now prove these.
Positively Scale-Invariant Flatness of ReLU Neural Networks
Venue: arXiv cs.LG 2019
Year: 2019

It was empirically confirmed by Keskar et al.\cite{SharpMinima} that flatter minima generalize better. However, for the popular ReLU network, sharp minimum can also generalize well \cite{SharpMinimacan}. The conclusion demonstrates that the existing definitions of flatness fail to account for the complex geometry of ReLU neural networks because they can't cover the Positively Scale-Invariant (PSI) property of ReLU network. In this paper, we formalize the PSI causes problem of existing definitions of flatness and propose a new description of flatness - \emph{PSI-flatness}. PSI-flatness is defined on the values of basis paths \cite{GSGD} instead of weights. Values of basis paths have been shown to be the PSI-variables and can sufficiently represent the ReLU neural networks which ensure the PSI property of PSI-flatness. Then we study the relation between PSI-flatness and generalization theoretically and empirically. First, we formulate a generalization bound based on PSI-flatness which shows generalization error decreasing with the ratio between the largest basis path value and the smallest basis path value. That is to say, the minimum with balanced values of basis paths will more likely to be flatter and generalize better. Finally. we visualize the PSI-flatness of loss surface around two learned models which indicates the minimum with smaller PSI-flatness can indeed generalize better.
Nonlinear input design as optimal control of a Hamiltonian system
Venue: arXiv cs.SY 2019
Year: 2019

We propose an input design method for a general class of parametric probabilistic models, including nonlinear dynamical systems with process noise. The goal of the procedure is to select inputs such that the parameter posterior distribution concentrates about the true value of the parameters; however, exact computation of the posterior is intractable. By representing (samples from) the posterior as trajectories from a certain Hamiltonian system, we transform the input design task into an optimal control problem. The method is illustrated via numerical examples, including MRI pulse sequence design.
Pinaka: Symbolic Execution meets Incremental Solving (Competition Contribution)
Venue: arXiv cs.LO 2019
Year: 2019

Many modern-day solvers offer functionality for incremental SAT solving, which preserves the state of the solver across invocations. This is beneficial when multiple, closely related SAT queries need to be fed to the solver. Pinaka is a symbolic execution engine which makes aggressive use of incremental SAT solving coupled with eager state infeasibility checks. It is built on top of the CProver/Symex framework. Pinaka supports both Breadth First Search and Depth First Search as state exploration strategies along with partial and full incremental modes. For SVCOMP 2019, Pinaka is configured to use partial incremental mode with Depth First Search strategy.
On the Succinctness of Atoms of Dependency
Venue: arXiv cs.LO 2019
Year: 2019

Propositional team logic is the propositional analog to first-order team logic. Non-classical atoms of dependence, independence, inclusion, exclusion and anonymity can be expressed in it, but for all atoms except dependence only exponential translations are known. In this paper, we systematically compare their succinctness in the existential fragment, where the splitting disjunction only occurs positively, and in full propositional team logic with unrestricted negation. By introducing a variant of the Ehrenfeucht-Fraïssé game called formula size game into team logic, we obtain exponential lower bounds in the existential fragment for all atoms. In the full fragment, we present polynomial upper bounds also for all atoms.
On the Optimality of Ali-Niesen Decentralized Coded Caching Scheme With and Without Error Correction
Venue: arXiv cs.IT 2019
Year: 2019

The decentralized coded caching was introduced in [M.~A. Maddah-Ali and U.~Niesen, `Decentralized coded caching attains order-optimal memory-rate tradeoff,' \emph{IEEE/ACM Trans. Networking}, vol.~23, no.~4, pp. 1029--1040, Aug. 2015] in which no coordination is required for the content placement. In this paper, with $N$ files and $K$ users, first it is shown that the decentralized coded caching scheme of Ali and Niesen is optimal for $N \geq K$ using techniques from index coding. Next the case when the shared link is error prone is considered. For this case, an optimal error correcting delivery scheme is proposed for coded caching problems with decentralized placement. Expression for the worst case rate is obtained when there are finite number of transmissions in error.
Performance Analysis of Effective Symbolic Methods for Solving Band Matrix SLAEs
Venue: arXiv cs.MS 2019
Year: 2019

This paper presents an experimental performance study of implementations of three symbolic algorithms for solving band matrix systems of linear algebraic equations with heptadiagonal, pentadiagonal, and tridiagonal coefficient matrices. The only assumption on the coefficient matrix in order for the algorithms to be stable is nonsingularity. These algorithms are implemented using the GiNaC library of C++ and the SymPy library of Python, considering five different data storing classes. Performance analysis of the implementations is done using the high-performance computing (HPC) platforms "HybriLIT" and "Avitohol". The experimental setup and the results from the conducted computations on the individual computer systems are presented and discussed. An analysis of the three algorithms is performed.
Fast Graph Representation Learning with PyTorch Geometric
Venue: arXiv cs.LG 2019
Year: 2019

We introduce PyTorch Geometric, a library for deep learning on irregularly structured input data such as graphs, point clouds and manifolds, built upon PyTorch. In addition to general graph data structures and processing methods, it contains a variety of recently published methods from the domains of relational learning and 3D data processing. PyTorch Geometric achieves high data throughput by leveraging sparse GPU acceleration, by providing dedicated CUDA kernels and by introducing efficient mini-batch handling for input examples of different size. In this work, we present the library in detail and perform a comprehensive comparative study of the implemented methods in homogeneous evaluation scenarios.
Safety-Guided Deep Reinforcement Learning via Online Gaussian Process Estimation
Venue: arXiv cs.LG 2019
Year: 2019

An important facet of reinforcement learning (RL) has to do with how the agent goes about exploring the environment. Traditional exploration strategies typically focus on efficiency and ignore safety. However, for practical applications, ensuring safety of the agent during exploration is crucial since performing an unsafe action or reaching an unsafe state could result in irreversible damage to the agent. The main challenge of safe exploration is that characterizing the unsafe states and actions is difficult for large continuous state or action spaces and unknown environments. In this paper, we propose a novel approach to incorporate estimations of safety to guide exploration and policy search in deep reinforcement learning. By using a cost function to capture trajectory-based safety, our key idea is to formulate the state-action value function of this safety cost as a candidate Lyapunov function and extend control-theoretic results to approximate its derivative using online Gaussian Process (GP) estimation. We show how to use these statistical models to guide the agent in unknown environments to obtain high-performance control policies with provable stability certificates.
Channel Decoding with Quantum Approximate Optimization Algorithm
Venue: arXiv cs.IT 2019
Year: 2019

Motivated by the recent advancement of quantum processors, we investigate quantum approximate optimization algorithm (QAOA) to employ quasi-maximum-likelihood (ML) decoding of classical channel codes. QAOA is a hybrid quantum-classical variational algorithm, which is advantageous for the near-term noisy intermediate-scale quantum (NISQ) devices, where the fidelity of quantum gates is limited by noise and decoherence. We first describe how to construct Ising Hamiltonian model to realize quasi-ML decoding with QAOA. For level-1 QAOA, we derive the systematic way to generate theoretical expressions of cost expectation for arbitrary binary linear codes. Focusing on (7, 4) Hamming code as an example, we analyze the impact of the degree distribution in associated generator matrix on the quantum decoding performance. The excellent performance of higher-level QAOA decoding is verified when Pauli rotation angles are optimized through meta-heuristic variational quantum eigensolver (VQE). Furthermore, we demonstrate the QAOA decoding performance in a real quantum device.
Softpressure: A Schedule-Driven Backpressure Algorithm for Coping with Network Congestion
Venue: arXiv cs.NI 2019
Year: 2019

We consider the problem of minimizing the delay of jobs moving through a directed graph of service nodes. In this problem, each node may have several links and is constrained to serve one link at a time. As jobs move through the network, they can pass through a node only after they have been serviced by that node. The objective is to minimize the delay jobs incur sitting in queues waiting to be serviced. Two distinct approaches to this problem have emerged from respective work in queuing theory and dynamic scheduling: the backpressure algorithm and schedule-driven control. In this paper, we present a hybrid approach of those two methods that incorporates the stability of queuing theory into a schedule-driven control framework. We then demonstrate how this hybrid method outperforms the other two in a real-time traffic signal control problem, where the nodes are traffic lights, the links are roads, and the jobs are vehicles. We show through simulations that, in scenarios with heavy congestion, the hybrid method results in 50% and 15% reductions in delay over schedule-driven control and backpressure respectively. A theoretical analysis also justifies our results.
Incremental Computation of Concept Diagrams
Venue: arXiv cs.LO 2019
Year: 2019

Suppose a formal context K=(G,M,I) is given, whose concept lattice B(K) with an attribute-additive concept diagram is already known, and an attribute column C=(G,{n},J) shall be inserted to or removed from it. This paper introduces and proves an incremental update algorithm for both tasks.
Continual Learning Using World Models for Pseudo-Rehearsal
Venue: arXiv cs.LG 2019
Year: 2019

The utility of learning a dynamics/world model of the environment in reinforcement learning has been shown in a many ways. When using neural networks, however, these models suffer catastrophic forgetting when learned in a lifelong or continual fashion. Current solutions to the continual learning problem require experience to be segmented and labeled as discrete tasks, however, in continuous experience it is generally unclear what a sufficient segmentation of tasks would be. Here we propose a method to continually learn these internal world models through the interleaving of internally generated episodes of past experiences (i.e., pseudo-rehearsal). We show this method can sequentially learn unsupervised temporal prediction, without task labels, in a disparate set of Atari games. Empirically, this interleaving of the internally generated rollouts with the external environment's observations leads to a consistent reduction in temporal prediction loss compared to non-interleaved learning and is preserved over repeated random exposures to various tasks. Similarly, using a network distillation approach, we show that modern policy gradient based reinforcement learning algorithms can use this internal model to continually learn to optimize reward based on the world model's representation of the environment.
Energy Efficient Optimization of Wireless-powered 5G Full Duplex Cellular Networks: A Mean Field Game Approach
Venue: arXiv cs.NI 2019
Year: 2019

This paper studies the power allocation of an ultra-dense cellular network consisting of multiple full-duplex (FD) base stations (BSs) serving a large number of half-duplex (HD) user equipments (UEs) located in a wide geographical area. Each BS consists of a baseband unit (BBU) that is responsible for signal processing and base station control, and a radio remote unit (RRU) that corresponds to a radio transceiver remotely built closer to the UEs. We consider a wireless-powered cellular network in which the BBU can periodically charge the RRU. We model the energy efficiency and coverage optimization problem for this network as a mean field game. We consider the weighted energy efficiency of the BS as the main performance metrics and evaluate the optimal strategy that can be adopted by the FD BSs in an ultra-dense network setting. Based on the interference and network energy efficiency models of the mean field game theory, Nash equilibrium of our proposed game is derived. Utilizing solutions of Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck-Kolmogorov (FPK) equations, a new power transmission strategy is developed for the BSs to optimize the energy efficiency of 5G cellular networks with full duplex transmissions. Simulation results indicate that the proposed strategy not only improves the energy efficiency but also ensures the average network coverage probability converges to a stable level.
CE-Net: Context Encoder Network for 2D Medical Image Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

Medical image segmentation is an important step in medical image analysis. With the rapid development of convolutional neural network in image processing, deep learning has been used for medical image segmentation, such as optic disc segmentation, blood vessel detection, lung segmentation, cell segmentation, etc. Previously, U-net based approaches have been proposed. However, the consecutive pooling and strided convolutional operations lead to the loss of some spatial information. In this paper, we propose a context encoder network (referred to as CE-Net) to capture more high-level information and preserve spatial information for 2D medical image segmentation. CE-Net mainly contains three major components: a feature encoder module, a context extractor and a feature decoder module. We use pretrained ResNet block as the fixed feature extractor. The context extractor module is formed by a newly proposed dense atrous convolution (DAC) block and residual multi-kernel pooling (RMP) block. We applied the proposed CE-Net to different 2D medical image segmentation tasks. Comprehensive results show that the proposed method outperforms the original U-Net method and other state-of-the-art methods for optic disc segmentation, vessel detection, lung segmentation, cell contour segmentation and retinal optical coherence tomography layer segmentation.
Stronger L2/L2 Compressed Sensing; Without Iterating
Venue: arXiv cs.DS 2019
Year: 2019

We consider the extensively studied problem of $\ell_2/\ell_2$ compressed sensing. The main contribution of our work is an improvement over [Gilbert, Li, Porat and Strauss, STOC 2010] with faster decoding time and significantly smaller column sparsity, answering two open questions of the aforementioned work. Previous work on sublinear-time compressed sensing employed an iterative procedure, recovering the heavy coordinates in phases. We completely depart from that framework, and give the first sublinear-time $\ell_2/\ell_2$ scheme which achieves the optimal number of measurements without iterating; this new approach is the key step to our progress. Towards that, we satisfy the $\ell_2/\ell_2$ guarantee by exploiting the heaviness of coordinates in a way that was not exploited in previous work. Via our techniques we obtain improved results for various sparse recovery tasks, and indicate possible further applications to problems in the field, to which the aforementioned iterative procedure creates significant obstructions.
Active and Semi-Supervised Learning in ASR: Benefits on the Acoustic and Language Models
Venue: arXiv cs.CL 2019
Year: 2019

The goal of this paper is to simulate the benefits of jointly applying active learning (AL) and semi-supervised training (SST) in a new speech recognition application. Our data selection approach relies on confidence filtering, and its impact on both the acoustic and language models (AM and LM) is studied. While AL is known to be beneficial to AM training, we show that it also carries out substantial improvements to the LM when combined with SST. Sophisticated confidence models, on the other hand, did not prove to yield any data selection gain. Our results indicate that, while SST is crucial at the beginning of the labeling process, its gains degrade rapidly as AL is set in place. The final simulation reports that AL allows a transcription cost reduction of about 70% over random selection. Alternatively, for a fixed transcription budget, the proposed approach improves the word error rate by about 12.5% relative.
A Scalable Max-Consensus Protocol For Noisy Ultra-Dense Networks
Venue: arXiv cs.IT 2019
Year: 2019

We introduce \emph{ScalableMax}, a novel communication scheme for achieving max-consensus in a network of multiple agents which harnesses the interference in the wireless channel as well as its multicast capabilities. In a sufficiently dense network, the amount of communication resources required grows logarithmically with the number of nodes, while in state-of-the-art approaches, this growth is at least linear. ScalableMax can handle additive noise and works well in a high SNR regime. For medium and low SNR, we propose the \emph{ScalableMax-EC} scheme, which extends the ideas of ScalableMax by introducing a novel error correction scheme. It achieves lower error rates at the cost of using more channel resources. However, it preserves the logarithmic growth with the number of agents in the system.