OpenRaiser/Intern-Atlas huggingface dataset:
Synchronized Detection and Recovery of Steganographic Messages with Adversarial Learning
Venue: arXiv cs.CV 2019
Year: 2019

In this work, we mainly study the mechanism of learning the steganographic algorithm as well as combining the learning process with adversarial learning to learn a good steganographic algorithm. To handle the problem of embedding secret messages into the specific medium, we design a novel adversarial modules to learn the steganographic algorithm, and simultaneously train three modules called generator, discriminator and steganalyzer. Different from existing methods, the three modules are formalized as a game to communicate with each other. In the game, the generator and discriminator attempt to communicate with each other using secret messages hidden in an image. While the steganalyzer attempts to analyze whether there is a transmission of confidential information. We show that through unsupervised adversarial training, the adversarial model can produce robust steganographic solutions, which act like an encryption. Furthermore, we propose to utilize supervised adversarial training method to train a robust steganalyzer, which is utilized to discriminate whether an image contains secret information. Numerous experiments are conducted on publicly available dataset to demonstrate the effectiveness of the proposed method.
Multi-Layer Competitive-Cooperative Framework for Performance Enhancement of Differential Evolution
Venue: arXiv cs.NE 2019
Year: 2019

Differential Evolution (DE) is recognized as one of the most powerful optimizers in the evolutionary algorithm (EA) family. Many DE variants were proposed in recent years, but significant differences in performances between them are hardly observed. Therefore, this paper suggests a multi-layer competitive-cooperative (MLCC) framework to facilitate the competition and cooperation of multiple DEs, which in turns, achieve a significant performance improvement. Unlike other multi-method strategies which adopt a multi-population based structure, with individuals only evolving in their corresponding subpopulations, MLCC implements a parallel structure with the entire population simultaneously monitored by multiple DEs assigned to their corresponding layers. An individual can store, utilize and update its evolution information in different layers based on an individual preference based layer selecting (IPLS) mechanism and a computational resource allocation bias (RAB) mechanism. In IPLS, individuals connect to only one favorite layer. While in RAB, high-quality solutions are evolved by considering all the layers. Thus DEs associated in the layers work in a competitive and cooperative manner. The proposed MLCC framework has been implemented on several highly competitive DEs. Experimental studies show that the MLCC variants significantly outperform the baseline DEs as well as several state-of-the-art and up-to-date DEs on CEC benchmark functions.
A group law on the projective plane with applications in Public Key Cryptography
Venue: arXiv cs.CR 2019
Year: 2019

We present a new group law defined on a subset of the projective plane $\mathbb{F}P^2$ over an arbitrary field $\mathbb{F}$, which lends itself to applications in Public Key Cryptography, in particular to a Diffie-Hellman-like key agreement protocol. We analyze the computational difficulty of solving the mathematical problem underlying the proposed Abelian group law and we prove that the security of our proposal is equivalent to the discrete logarithm problem in the multiplicative group of the cubic extension of the finite field considered. Finally, we present a variant of the proposed group law but over the ring $\mathbb{Z}/pq\mathbb{Z}$, and explain how the security becomes enhanced, though at the cost of a longer key length.
VR-Goggles for Robots: Real-to-sim Domain Adaptation for Visual Control
Venue: arXiv cs.RO 2019
Year: 2019

In this paper, we deal with the reality gap from a novel perspective, targeting transferring Deep Reinforcement Learning (DRL) policies learned in simulated environments to the real-world domain for visual control tasks. Instead of adopting the common solutions to the problem by increasing the visual fidelity of synthetic images output from simulators during the training phase, we seek to tackle the problem by translating the real-world image streams back to the synthetic domain during the deployment phase, to make the robot feel at home. We propose this as a lightweight, flexible, and efficient solution for visual control, as 1) no extra transfer steps are required during the expensive training of DRL agents in simulation; 2) the trained DRL agents will not be constrained to being deployable in only one specific real-world environment; 3) the policy training and the transfer operations are decoupled, and can be conducted in parallel. Besides this, we propose a simple yet effective shift loss that is agnostic to the downstream task, to constrain the consistency between subsequent frames which is important for consistent policy outputs. We validate the shift loss for artistic style transfer for videos and domain adaptation, and validate our visual control approach in indoor and outdoor robotics experiments.
Nearly Optimal Dynamic $k$-Means Clustering for High-Dimensional Data
Venue: arXiv cs.DS 2019
Year: 2019

We consider the $k$-means clustering problem in the dynamic streaming setting, where points from a discrete Euclidean space $\{1, 2, \ldots, Δ\}^d$ can be dynamically inserted to or deleted from the dataset. For this problem, we provide a one-pass coreset construction algorithm using space $\tilde{O}(k\cdot \mathrm{poly}(d, \logΔ))$, where $k$ is the target number of centers. To our knowledge, this is the first dynamic geometric data stream algorithm for $k$-means using space polynomial in dimension and nearly optimal (linear) in $k$.
When can $l_p$-norm objective functions be minimized via graph cuts?
Venue: arXiv cs.DS 2019
Year: 2019

Techniques based on minimal graph cuts have become a standard tool for solving combinatorial optimization problems arising in image processing and computer vision applications. These techniques can be used to minimize objective functions written as the sum of a set of unary and pairwise terms, provided that the objective function is submodular. This can be interpreted as minimizing the $l_1$-norm of the vector containing all pairwise and unary terms. By raising each term to a power $p$, the same technique can also be used to minimize the $l_p$-norm of the vector. Unfortunately, the submodularity of an $l_1$-norm objective function does not guarantee the submodularity of the corresponding $l_p$-norm objective function. The contribution of this paper is to provide useful conditions under which an $l_p$-norm objective function is submodular for all $p\geq 1$, thereby identifying a large class of $l_p$-norm objective functions that can be minimized via minimal graph cuts.
A Generative Model for Natural Sounds Based on Latent Force Modelling
Venue: arXiv cs.LG 2019
Year: 2019

Recent advances in analysis of subband amplitude envelopes of natural sounds have resulted in convincing synthesis, showing subband amplitudes to be a crucial component of perception. Probabilistic latent variable analysis is particularly revealing, but existing approaches don't incorporate prior knowledge about the physical behaviour of amplitude envelopes, such as exponential decay and feedback. We use latent force modelling, a probabilistic learning paradigm that incorporates physical knowledge into Gaussian process regression, to model correlation across spectral subband envelopes. We augment the standard latent force model approach by explicitly modelling correlations over multiple time steps. Incorporating this prior knowledge strengthens the interpretation of the latent functions as the source that generated the signal. We examine this interpretation via an experiment which shows that sounds generated by sampling from our probabilistic model are perceived to be more realistic than those generated by similar models based on nonnegative matrix factorisation, even in cases where our model is outperformed from a reconstruction error perspective.
Sense-and-Predict: Harnessing Spatial Interference Correlation for Cognitive Radio Networks
Venue: arXiv cs.IT 2019
Year: 2019

Cognitive radio (CR) is a key enabler realizing future networks to achieve higher spectral efficiency by allowing spectrum sharing between different wireless networks. It is important to explore whether spectrum access opportunities are available, but conventional CR based on transmitter (TX) sensing cannot be used to this end because the paired receiver (RX) may experience different levels of interference, according to the extent of their separation, blockages, and beam directions. To address this problem, this paper proposes a novel form of medium access control (MAC) termed sense-and-predict (SaP), whereby each secondary TX predicts the interference level at the RX based on the sensed interference at the TX; this can be quantified in terms of a spatial interference correlation between the two locations. Using stochastic geometry, the spatial interference correlation can be expressed in the form of a conditional coverage probability, such that the signal-to-interference ratio (SIR) at the RX is no less than a predetermined threshold given the sensed interference at the TX, defined as an opportunistic probability (OP). The secondary TX randomly accesses the spectrum depending on OP. We optimize the SaP framework to maximize the area spectral efficiencies (ASEs) of secondary networks while guaranteeing the service quality of the primary networks. Testbed experiments using USRP and MATLAB simulations show that SaP affords higher ASEs compared with CR without prediction.
Repeat-Accumulate Signal Codes
Venue: arXiv cs.IT 2019
Year: 2019

State-constrained signal codes directly encode modulation signals using signal processing filters, the coefficients of which are constrained over the rings of formal power series. Although the performance of signal codes is defined by these signal filters, optimal filters must be found by brute-force search in terms of symbol error rate because the asymptotic behavior with different filters has not been investigated. Moreover, computational complexity of the conventional BCJR used in the decoder increases exponentially as the number of output constellations increase. We hence propose a new class of state-constrained signal codes called repeat-accumulate signal codes (RASCs). To analyze the asymptotic behavior of these codes, we employ Monte Carlo density evolution (MC-DE). As a result, the optimum filters can be efficiently found for given parameters of the encoder. We also introduce a low-complexity decoding algorithm for RASCs called the extended min-sum (EMS) decoder. The MC-DE analysis shows that the difference between noise thresholds of RASC and the Shannon limit is within 0.8 dB. Simulation results moreover show that the EMS decoder can reduce the computational complexity to less than 25 % of that of conventional decoder without degrading the performance by more than 1 dB.
Deep Learning-based Channel Estimation for Beamspace mmWave Massive MIMO Systems
Venue: arXiv cs.IT 2019
Year: 2019

Channel estimation is very challenging when the receiver is equipped with a limited number of radio-frequency (RF) chains in beamspace millimeter-wave (mmWave) massive multiple-input and multiple-output systems. To solve this problem, we exploit a learned denoising-based approximate message passing (LDAMP) network. This neural network can learn channel structure and estimate channel from a large number of training data. Furthermore, we provide an analytical framework on the asymptotic performance of the channel estimator. Based on our analysis and simulation results, the LDAMP neural network significantly outperforms state-of-the-art compressed sensingbased algorithms even when the receiver is equipped with a small number of RF chains. Therefore, deep learning is a powerful tool for channel estimation in mmWave communications.
A Signature-based Algorithm for computing Computing Gröbner Bases over Principal Ideal Domains
Venue: arXiv cs.SC 2019
Year: 2019

Signature-based algorithms have become a standard approach for Gröbner basis computations for polynomial systems over fields, but how to extend these techniques to coefficients in general rings is not yet as well understood. In this paper, we present a proof-of-concept signature-based algorithm for computing Gröbner bases over commutative integral domains. It is adapted from a general version of Möller's algorithm (1988) which considers reductions by multiple polynomials at each step. This algorithm performs reductions with non-decreasing signatures, and in particular, signature drops do not occur. When the coefficients are from a principal ideal domain (e.g. the ring of integers or the ring of univariate polynomials over a field), we prove correctness and termination of the algorithm, and we show how to use signature properties to implement classic signature-based criteria to eliminate some redundant reductions. In particular, if the input is a regular sequence, the algorithm operates without any reduction to 0. We have written a toy implementation of the algorithm in Magma. Early experimental results suggest that the algorithm might even be correct and terminate in a more general setting, for polynomials over a unique factorization domain (e.g. the ring of multivariate polynomials over a field or a PID).
The Sea Exploration Problem: Data-driven Orienteering on a Continuous Surface
Venue: arXiv cs.AI 2019
Year: 2019

This paper describes a problem arising in sea exploration, where the aim is to schedule the expedition of a ship for collecting information about the resources on the seafloor. The aim is to collect data by probing on a set of carefully chosen locations, so that the information available is optimally enriched. This problem has similarities with the orienteering problem, where the aim is to plan a time-limited trip for visiting a set of vertices, collecting a prize at each of them, in such a way that the total value collected is maximum. In our problem, the score at each vertex is associated with an estimation of the level of the resource on the given surface, which is done by regression using Gaussian processes. Hence, there is a correlation among scores on the selected vertices; this is a first difference with respect to the standard orienteering problem. The second difference is the location of each vertex, which in our problem is a freely chosen point on a given surface.
Near-Optimal Coresets of Kernel Density Estimates
Venue: arXiv cs.LG 2019
Year: 2019

We construct near-optimal coresets for kernel density estimates for points in $\mathbb{R}^d$ when the kernel is positive definite. Specifically we show a polynomial time construction for a coreset of size $O(\sqrt{d}/\varepsilon\cdot \sqrt{\log 1/\varepsilon} )$, and we show a near-matching lower bound of size $Ω(\min\{\sqrt{d}/\varepsilon, 1/\varepsilon^2\})$. When $d\geq 1/\varepsilon^2$, it is known that the size of coreset can be $O(1/\varepsilon^2)$. The upper bound is a polynomial-in-$(1/\varepsilon)$ improvement when $d \in [3,1/\varepsilon^2)$ and the lower bound is the first known lower bound to depend on $d$ for this problem. Moreover, the upper bound restriction that the kernel is positive definite is significant in that it applies to a wide-variety of kernels, specifically those most important for machine learning. This includes kernels for information distances and the sinc kernel which can be negative.
Caregiver Assessment Using Smart Gaming Technology: A Preliminary Approach
Venue: arXiv cs.HC 2019
Year: 2019

As pre-diagnostic technologies are becoming increasingly accessible, using them to improve the quality of care available to dementia patients and their caregivers is of increasing interest. Specifically, we aim to develop a tool for non-invasively assessing task performance in a simple gaming application. To address this, we have developed Caregiver Assessment using Smart Gaming Technology (CAST), a mobile application that personalizes a traditional word scramble game. Its core functionality uses a Fuzzy Inference System (FIS) optimized via a Genetic Algorithm (GA) to provide customized performance measures for each user of the system. With CAST, we match the relative level of difficulty of play using the individual's ability to solve the word scramble tasks. We provide an analysis of the preliminary results for determining task difficulty, with respect to our current participant cohort.
Reinforcement Learning with Wasserstein Distance Regularisation, with Applications to Multipolicy Learning
Venue: arXiv cs.LG 2019
Year: 2019

We describe an application of Wasserstein distance to Reinforcement Learning. The Wasserstein distance in question is between the distribution of mappings of trajectories of a policy into some metric space, and some other fixed distribution (which may, for example, come from another policy). Different policies induce different distributions, so given an underlying metric, the Wasserstein distance quantifies how different policies are. This can be used to learn multiple polices which are different in terms of such Wasserstein distances by using a Wasserstein regulariser. Changing the sign of the regularisation parameter, one can learn a policy for which its trajectory mapping distribution is attracted to a given fixed distribution.
Junction Tree Variational Autoencoder for Molecular Graph Generation
Venue: arXiv cs.LG 2019
Year: 2019

We seek to automate the design of molecules based on specific chemical properties. In computational terms, this task involves continuous embedding and generation of molecular graphs. Our primary contribution is the direct realization of molecular graphs, a task previously approached by generating linear SMILES strings instead of graphs. Our junction tree variational autoencoder generates molecular graphs in two phases, by first generating a tree-structured scaffold over chemical substructures, and then combining them into a molecule with a graph message passing network. This approach allows us to incrementally expand molecules while maintaining chemical validity at every step. We evaluate our model on multiple tasks ranging from molecular generation to optimization. Across these tasks, our model outperforms previous state-of-the-art baselines by a significant margin.
Adversarially Regularized Graph Autoencoder for Graph Embedding
Venue: arXiv cs.LG 2019
Year: 2019

Graph embedding is an effective method to represent graph data in a low dimensional space for graph analytics. Most existing embedding algorithms typically focus on preserving the topological structure or minimizing the reconstruction errors of graph data, but they have mostly ignored the data distribution of the latent codes from the graphs, which often results in inferior embedding in real-world graph data. In this paper, we propose a novel adversarial graph embedding framework for graph data. The framework encodes the topological structure and node content in a graph to a compact representation, on which a decoder is trained to reconstruct the graph structure. Furthermore, the latent representation is enforced to match a prior distribution via an adversarial training scheme. To learn a robust embedding, two variants of adversarial approaches, adversarially regularized graph autoencoder (ARGA) and adversarially regularized variational graph autoencoder (ARVGA), are developed. Experimental studies on real-world graphs validate our design and demonstrate that our algorithms outperform baselines by a wide margin in link prediction, graph clustering, and graph visualization tasks.
Improving Quadrature for Constrained Integrands
Venue: arXiv cs.LG 2019
Year: 2019

We present an improved Bayesian framework for performing inference of affine transformations of constrained functions. We focus on quadrature with nonnegative functions, a common task in Bayesian inference. We consider constraints on the range of the function of interest, such as nonnegativity or boundedness. Although our framework is general, we derive explicit approximation schemes for these constraints, and argue for the use of a log transformation for functions with high dynamic range such as likelihood surfaces. We propose a novel method for optimizing hyperparameters in this framework: we optimize the marginal likelihood in the original space, as opposed to in the transformed space. The result is a model that better explains the actual data. Experiments on synthetic and real-world data demonstrate our framework achieves superior estimates using less wall-clock time than existing Bayesian quadrature procedures.
Isolating Sources of Disentanglement in Variational Autoencoders
Venue: arXiv cs.LG 2019
Year: 2019

We decompose the evidence lower bound to show the existence of a term measuring the total correlation between latent variables. We use this to motivate our $β$-TCVAE (Total Correlation Variational Autoencoder), a refinement of the state-of-the-art $β$-VAE objective for learning disentangled representations, requiring no additional hyperparameters during training. We further propose a principled classifier-free measure of disentanglement called the mutual information gap (MIG). We perform extensive quantitative and qualitative experiments, in both restricted and non-restricted settings, and show a strong relation between total correlation and disentanglement, when the latent variables model is trained using our framework.
Putting a bug in ML: The moth olfactory network learns to read MNIST
Venue: arXiv cs.LG 2019
Year: 2019

We seek to (i) characterize the learning architectures exploited in biological neural networks for training on very few samples, and (ii) port these algorithmic structures to a machine learning context. The Moth Olfactory Network is among the simplest biological neural systems that can learn, and its architecture includes key structural elements and mechanisms widespread in biological neural nets, such as cascaded networks, competitive inhibition, high intrinsic noise, sparsity, reward mechanisms, and Hebbian plasticity. These structural biological elements, in combination, enable rapid learning. MothNet is a computational model of the Moth Olfactory Network, closely aligned with the moth's known biophysics and with in vivo electrode data collected from moths learning new odors. We assign this model the task of learning to read the MNIST digits. We show that MothNet successfully learns to read given very few training samples (1 to 10 samples per class). In this few-samples regime, it outperforms standard machine learning methods such as nearest-neighbors, support-vector machines, and neural networks (NNs), and matches specialized one-shot transfer-learning methods but without the need for pre-training. The MothNet architecture illustrates how algorithmic structures derived from biological brains can be used to build alternative NNs that may avoid some of the learning rate limitations of current engineered NNs.
Models of Type Theory Based on Moore Paths
Venue: arXiv cs.LO 2019
Year: 2019

This paper introduces a new family of models of intensional Martin-Löf type theory. We use constructive ordered algebra in toposes. Identity types in the models are given by a notion of Moore path. By considering a particular gros topos, we show that there is such a model that is non-truncated, i.e. contains non-trivial structure at all dimensions. In other words, in this model a type in a nested sequence of identity types can contain more than one element, no matter how great the degree of nesting. Although inspired by existing non-truncated models of type theory based on simplicial and cubical sets, the notion of model presented here is notable for avoiding any form of Kan filling condition in the semantics of types.