OpenRaiser/Intern-Atlas huggingface dataset:
Breaking Speech Recognizers to Imagine Lyrics
Venue: arXiv cs.HC 2019
Year: 2019

We introduce a new method for generating text, and in particular song lyrics, based on the speech-like acoustic qualities of a given audio file. We repurpose a vocal source separation algorithm and an acoustic model trained to recognize isolated speech, instead inputting instrumental music or environmental sounds. Feeding the "mistakes" of the vocal separator into the recognizer, we obtain a transcription of words \emph{imagined} to be spoken in the input audio. We describe the key components of our approach, present initial analysis, and discuss the potential of the method for machine-in-the-loop collaboration in creative applications.
A Shape Transformation-based Dataset Augmentation Framework for Pedestrian Detection
Venue: arXiv cs.CV 2019
Year: 2019

Deep learning-based computer vision is usually data-hungry. Many researchers attempt to augment datasets with synthesized data to improve model robustness. However, the augmentation of popular pedestrian datasets, such as Caltech and Citypersons, can be extremely challenging because real pedestrians are commonly in low quality. Due to the factors like occlusions, blurs, and low-resolution, it is significantly difficult for existing augmentation approaches, which generally synthesize data using 3D engines or generative adversarial networks (GANs), to generate realistic-looking pedestrians. Alternatively, to access much more natural-looking pedestrians, we propose to augment pedestrian detection datasets by transforming real pedestrians from the same dataset into different shapes. Accordingly, we propose the Shape Transformation-based Dataset Augmentation (STDA) framework. The proposed framework is composed of two subsequent modules, i.e. the shape-guided deformation and the environment adaptation. In the first module, we introduce a shape-guided warping field to help deform the shape of a real pedestrian into a different shape. Then, in the second stage, we propose an environment-aware blending map to better adapt the deformed pedestrians into surrounding environments, obtaining more realistic-looking pedestrians and more beneficial augmentation results for pedestrian detection. Extensive empirical studies on different pedestrian detection benchmarks show that the proposed STDA framework consistently produces much better augmentation results than other pedestrian synthesis approaches using low-quality pedestrians. By augmenting the original datasets, our proposed framework also improves the baseline pedestrian detector by up to 38% on the evaluated benchmarks, achieving state-of-the-art performance.
How Should an Agent Practice?
Venue: arXiv cs.AI 2019
Year: 2019

We present a method for learning intrinsic reward functions to drive the learning of an agent during periods of practice in which extrinsic task rewards are not available. During practice, the environment may differ from the one available for training and evaluation with extrinsic rewards. We refer to this setup of alternating periods of practice and objective evaluation as practice-match, drawing an analogy to regimes of skill acquisition common for humans in sports and games. The agent must effectively use periods in the practice environment so that performance improves during matches. In the proposed method the intrinsic practice reward is learned through a meta-gradient approach that adapts the practice reward parameters to reduce the extrinsic match reward loss computed from matches. We illustrate the method on a simple grid world, and evaluate it in two games in which the practice environment differs from match: Pong with practice against a wall without an opponent, and PacMan with practice in a maze without ghosts. The results show gains from learning in practice in addition to match periods over learning in matches only.
Computational Induction of Prosodic Structure
Venue: arXiv cs.CL 2019
Year: 2019

The present study has two goals relating to the grammar of prosody, understood as the rhythms and melodies of speech. First, an overview is provided of the computable grammatical and phonetic approaches to prosody analysis which use hypothetico-deductive methods and are based on learned hermeneutic intuitions about language. Second, a proposal is presented for an inductive grounding in the physical signal, in which prosodic structure is inferred using a language-independent method from the low-frequency spectrum of the speech signal. The overview includes a discussion of computational aspects of standard generative and post-generative models, and suggestions for reformulating these to form inductive approaches. Also included is a discussion of linguistic phonetic approaches to analysis of annotations (pairs of speech unit labels with time-stamps) of recorded spoken utterances. The proposal introduces the inductive approach of Rhythm Formant Theory (RFT) and the associated Rhythm Formant Analysis (RFA) method are introduced, with the aim of completing a gap in the linguistic hypothetico-deductive cycle by grounding in a language-independent inductive procedure of speech signal analysis. The validity of the method is demonstrated and applied to rhythm patterns in read-aloud Mandarin Chinese, finding differences from English which are related to lexical and grammatical differences between the languages, as well as individual variation. The overall conclusions are (1) that normative language-to-language phonological or phonetic comparisons of rhythm, for example of Mandarin and English, are too simplistic, in view of diverse language-internal factors due to genre and style differences as well as utterance dynamics, and (2) that language-independent empirical grounding of rhythm in the physical signal is called for.
Minimizing Age of Information for Real-Time Monitoring in Resource-Constrained Industrial IoT Networks
Venue: arXiv cs.IT 2019
Year: 2019

This paper considers an Industrial Internet of Thing (IIoT) system with a source monitoring a dynamic process with randomly generated status updates. The status updates are sent to an designated destination in a real-time manner over an unreliable link. The source is subject to a practical constraint of limited average transmission power. Thus, the system should carefully schedule when to transmit a fresh status update or retransmit the stale one. To characterize the performance of timely status update, we adopt a recent concept, Age of Information (AoI), as the performance metric. We aim to minimize the long-term average AoI under the limited average transmission power at the source, by formulating a constrained Markov Decision Process (CMDP) problem. To address the formulated CMDP, we recast it into an unconstrained Markov Decision Process (MDP) through Lagrangian relaxation. We prove the existence of optimal stationary policy of the original CMDP, which is a randomized mixture of two deterministic stationary policies of the unconstrained MDP. We also explore the characteristics of the problem to reduce the action space of each state to significantly reduce the computation complexity. We further prove the threshold structure of the optimal deterministic policy for the unconstrained MDP. Simulation results show the proposed optimal policy achieves lower average AoI compared with random policy, especially when the system suffers from stricter resource constraint. Besides, the influence of status generation probability and transmission failure rate on optimal policy and the resultant average AoI as well as the impact of average transmission power on the minimal average AoI are unveiled.
Towards Omni-Supervised Face Alignment for Large Scale Unlabeled Videos
Venue: arXiv cs.CV 2019
Year: 2019

In this paper, we propose a spatial-temporal relational reasoning networks (STRRN) approach to investigate the problem of omni-supervised face alignment in videos. Unlike existing fully supervised methods which rely on numerous annotations by hand, our learner exploits large scale unlabeled videos plus available labeled data to generate auxiliary plausible training annotations. Motivated by the fact that neighbouring facial landmarks are usually correlated and coherent across consecutive frames, our approach automatically reasons about discriminative spatial-temporal relationships among landmarks for stable face tracking. Specifically, we carefully develop an interpretable and efficient network module, which disentangles facial geometry relationship for every static frame and simultaneously enforces the bi-directional cycle-consistency across adjacent frames, thus allowing the modeling of intrinsic spatial-temporal relations from raw face sequences. Extensive experimental results demonstrate that our approach surpasses the performance of most fully supervised state-of-the-arts.
Evaluating the Vulnerability of Communities in Social Networks by Gravity Model
Venue: arXiv cs.SI 2019
Year: 2019

With the development of network science, the various properties of complex networks have recently received extensive attention. Among these properties, the vulnerability of the communities (VoCs) is particularly important. In the conventional research, only parts of structural features of the community rather than multiple aspects are considered in the evaluating model. However, in reality, the impact on the VoC is multifaceted, not only its own structure property, but also the influence of other communities. In order to better model the influence between communities, so as to evaluate the VoCs in the social network, a gravity-based community vulnerability evaluation model is proposed in this paper. In this proposed model, three different aspects of the factor are considered, i.e. the number of edges inside the community, the number of edges connected neighboring communities, and the gravity index (GI) of each community, which correspond to the interior information, small scale interaction relationship, and large scale interaction relationship of communities. By means of the Jensen-Shannon divergence (JSD) and log-sigmoid transition (LST) function, the abstract distance (AD) between each pair of communities can be calculated to construct the community network (CN). With the usage of gravity model, the GI of each community which describes the large scale interaction relationship can be obtained. Eventually, the community vulnerability degree and order can be calculated by this proposed model, and the sensitivity of weighting parameters is analyzed by Sobol' indices. In particular, this proposed method can degenerate to the classical method with the setting of weighting parameters. The effectiveness and reasonability of this proposed model are demonstrated by several real world complex networks.
MetaFusion: Controlled False-Negative Reduction of Minority Classes in Semantic Segmentation
Venue: arXiv cs.CV 2019
Year: 2019

In semantic segmentation datasets, classes of high importance are oftentimes underrepresented, e.g., humans in street scenes. Neural networks are usually trained to reduce the overall number of errors, attaching identical loss to errors of all kinds. However, this is not necessarily aligned with human intuition. For instance, an overlooked pedestrian seems more severe than an incorrectly detected one. One possible remedy is to deploy different decision rules by introducing class priors which assigns larger weight to underrepresented classes. While reducing the false-negatives of the underrepresented class, at the same time this leads to a considerable increase of false-positive indications. In this work, we combine decision rules with methods for false-positive detection. We therefore fuse false-negative detection with uncertainty based false-positive meta classification. We present proof-of-concept results for CIFAR-10, and prove the efficiency of our method for the semantic segmentation of street scenes on the Cityscapes dataset based on predicted instances of the 'human' class. In the latter we employ an advanced false-positive detection method using uncertainty measures aggregated over instances. We thereby achieve improved trade-offs between false-negative and false-positive samples of the underrepresented classes.
VarNet: Variational Neural Networks for the Solution of Partial Differential Equations
Venue: arXiv cs.LG 2019
Year: 2019

In this paper we propose a new model-based unsupervised learning method, called VarNet, for the solution of partial differential equations (PDEs) using deep neural networks (NNs). Particularly, we propose a novel loss function that relies on the variational (integral) form of PDEs as apposed to their differential form which is commonly used in the literature. Our loss function is discretization-free, highly parallelizable, and more effective in capturing the solution of PDEs since it employs lower-order derivatives and trains over measure non-zero regions of space-time. Given this loss function, we also propose an approach to optimally select the space-time samples, used to train the NN, that is based on the feedback provided from the PDE residual. The models obtained using VarNet are smooth and do not require interpolation. They are also easily differentiable and can directly be used for control and optimization of PDEs. Finally, VarNet can straight-forwardly incorporate parametric PDE models making it a natural tool for model order reduction (MOR) of PDEs. We demonstrate the performance of our method through extensive numerical experiments for the advection-diffusion PDE as an important case-study.
Realization of spatial sparseness by deep ReLU nets with massive data
Venue: arXiv cs.LG 2019
Year: 2019

The great success of deep learning poses urgent challenges for understanding its working mechanism and rationality. The depth, structure, and massive size of the data are recognized to be three key ingredients for deep learning. Most of the recent theoretical studies for deep learning focus on the necessity and advantages of depth and structures of neural networks. In this paper, we aim at rigorous verification of the importance of massive data in embodying the out-performance of deep learning. To approximate and learn spatially sparse and smooth functions, we establish a novel sampling theorem in learning theory to show the necessity of massive data. We then prove that implementing the classical empirical risk minimization on some deep nets facilitates in realization of the optimal learning rates derived in the sampling theorem. This perhaps explains why deep learning performs so well in the era of big data.
Polynomial Rewritings from Expressive Description Logics with Closed Predicates to Variants of Datalog
Venue: arXiv cs.AI 2019
Year: 2019

In many scenarios, complete and incomplete information coexist. For this reason, the knowledge representation and database communities have long shown interest in simultaneously supporting the closed- and the open-world views when reasoning about logic theories. Here we consider the setting of querying possibly incomplete data using logic theories, formalized as the evaluation of an ontology-mediated query (OMQ) that pairs a query with a theory, sometimes called an ontology, expressing background knowledge. This can be further enriched by specifying a set of closed predicates from the theory that are to be interpreted under the closed-world assumption, while the rest are interpreted with the open-world view. In this way we can retrieve more precise answers to queries by leveraging the partial completeness of the data. The central goal of this paper is to understand the relative expressiveness of OMQ languages in which the ontology is written in the expressive Description Logic (DL) ALCHOI and includes a set of closed predicates. We consider a restricted class of conjunctive queries. Our main result is to show that every query in this non-monotonic query language can be translated in polynomial time into Datalog with negation under the stable model semantics. To overcome the challenge that Datalog has no direct means to express the existential quantification present in ALCHOI, we define a two-player game that characterizes the satisfaction of the ontology, and design a Datalog query that can decide the existence of a winning strategy for the game. If there are no closed predicates, that is in the case of querying a plain ALCHOI knowledge base, our translation yields a positive disjunctive Datalog program of polynomial size. To the best of our knowledge, unlike previous translations for related fragments with expressive (non-Horn) DLs, these are the first polynomial time translations.
Cyber Physical Systems (CPS) Surveillance Using An Epidemic Model
Venue: arXiv cs.DC 2019
Year: 2019

Vast investments have recently been made worldwide in developing the Cyber-Physical System (CPS) technology with the expectations of improving economical and societal structures. However, great care must be paid to the CPS' complexity, the impact of emerging IoT (Internet of Things) protocols on the CPS infrastructure as well as the impact of information dissemination by these protocols on the safety of these infrastructures. This paper addresses the issue of CPS safety by proposing and evaluating the performance of a CPS management framework and the analysis of the dynamics of the underlining IoT network in the cyber-space. The main contributions of this paper are in threefold. Firstly, a new CPS framework is proposed; that: 1) builds around a layered architecture to compartmentalise the CPS functionalities into different modules for efficiency and scalability and 2) uses an inner feedback loop for the efficient management of CPS infrastructure. Secondly, building upon this framework, a novel diffusion model that uses the epidemic (interference) sets to produce accurate diffusion patterns across the CPS IoT subsystem is proposed. Finally, the proposed diffusion model is numerically analysed to show how it can be used to achieve efficient CPS surveillance in order to trigger reconfiguration to re-optimise the CPS when it is under stress. in IoT settings. The numerical analysis of the diffusion model shows that interference propagates in pairwise disjoint sets, with IoT nodes migrating from "susceptible" to "attacked" statuses and finally reaching the "removed" state at a predictable time. Deployment considerations on some of the current social and public networks are also onsidered
Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments
Venue: arXiv cs.DS 2019
Year: 2019

We consider the problem of learning a mixture of linear regressions (MLRs). An MLR is specified by $k$ nonnegative mixing weights $p_1, \ldots, p_k$ summing to $1$, and $k$ unknown regressors $w_1,...,w_k\in\mathbb{R}^d$. A sample from the MLR is drawn by sampling $i$ with probability $p_i$, then outputting $(x, y)$ where $y = \langle x, w_i \rangle + η$, where $η\sim\mathcal{N}(0,ς^2)$ for noise rate $ς$. Mixtures of linear regressions are a popular generative model and have been studied extensively in machine learning and theoretical computer science. However, all previous algorithms for learning the parameters of an MLR require running time and sample complexity scaling exponentially with $k$. In this paper, we give the first algorithm for learning an MLR that runs in time which is sub-exponential in $k$. Specifically, we give an algorithm which runs in time $\widetilde{O}(d)\cdot\exp(\widetilde{O}(\sqrt{k}))$ and outputs the parameters of the MLR to high accuracy, even in the presence of nontrivial regression noise. We demonstrate a new method that we call "Fourier moment descent" which uses univariate density estimation and low-degree moments of the Fourier transform of suitable univariate projections of the MLR to iteratively refine our estimate of the parameters. To the best of our knowledge, these techniques have never been used in the context of high dimensional distribution learning, and may be of independent interest. We also show that our techniques can be used to give a sub-exponential time algorithm for learning mixtures of hyperplanes, a natural hard instance of the subspace clustering problem.
Complexity-Adjustable SC Decoding of Polar Codes for Energy Consumption Reduction
Venue: arXiv cs.IT 2019
Year: 2019

This paper proposes an enhanced list-aided successive cancellation stack (ELSCS) decoding algorithm with adjustable decoding complexity. In addition, a logarithmic likelihood ratio (LLR)-threshold based path extension scheme is designed to further reduce the memory consumption of stack decoding. Numerical simulation results show that without affecting the error correction performance, the proposed ELSCS decoding algorithm provides a flexible tradeoff between time complexity and computational complexity, while reducing storage space up to $70\%$. Based on the fact that most mobile devices operate in environments with stringent energy budget to support diverse applications, the proposed scheme is a promising candidate for meeting requirements of different applications while maintaining a low computational complexity and computing resource utilization.
Evolution of Robust High Speed Optical-Flow-Based Landing for Autonomous MAVs
Venue: arXiv cs.RO 2019
Year: 2019

Automatic optimization of robotic behavior has been the long-standing goal of Evolutionary Robotics. Allowing the problem at hand to be solved by automation often leads to novel approaches and new insights. A common problem encountered with this approach is that when this optimization occurs in a simulated environment, the optimized policies are subject to the reality gap when implemented in the real world. This often results in sub-optimal behavior, if it works at all. This paper investigates the automatic optimization of neurocontrollers to perform quick but safe landing maneuvers for a quadrotor micro air vehicle using the divergence of the optical flow field of a downward looking camera. The optimized policies showed that a piece-wise linear control scheme is more effective than the simple linear scheme commonly used, something not yet considered by human designers. Additionally, we show the utility in using abstraction on the input and output of the controller as a tool to improve the robustness of the optimized policies to the reality gap by testing our policies optimized in simulation on real world vehicles. We tested the neurocontrollers using two different methods to generate and process the visual input, one using a conventional CMOS camera and one a dynamic vision sensor, both of which perform significantly differently than the simulated sensor. The use of the abstracted input resulted in near seamless transfer to the real world with the controllers showing high robustness to a clear reality gap.
Dim the Lights! -- Low-Rank Prior Temporal Data for Specular-Free Video Recovery
Venue: arXiv cs.CV 2019
Year: 2019

The appearance of an object is significantly affected by the illumination conditions in the environment. This is more evident with strong reflective objects as they suffer from more dominant specular reflections, causing information loss and discontinuity in the image domain. In this paper, we present a novel framework for specular-free video recovery with special emphasis on dealing with complex motions coming from objects or camera. Our solution is a twostep approach that allows for both detection and restoration of the damaged regions on video data. We first propose a spatially adaptive detection term that searches for the damage areas. We then introduce a variational solution for specular-free video recovery that allows exploiting spatio-temporal correlations by representing prior data in a low-rank form. We demonstrate that our solution prevents major drawbacks of existing approaches while improving the performance in both detection accuracy and inpainting quality. Finally, we show that our approach can be applied to other problems such as object removal.