pwc-archive/papers-with-abstracts huggingface dataset:
Active learning for enumerating local minima based on Gaussian process derivatives
Authors: Yu Inatsu, Daisuke Sugita, Kazuaki Toyoura, Ichiro Takeuchi
Date: 2019-03-08 00:00:00
Tasks: Active Learning, Gaussian Processes

We study active learning (AL) based on Gaussian Processes (GPs) for
efficiently enumerating all of the local minimum solutions of a black-box
function. This problem is challenging due to the fact that local solutions are
characterized by their zero gradient and positive-definite Hessian properties,
but those derivatives cannot be directly observed. We propose a new AL method
in which the input points are sequentially selected such that the confidence
intervals of the GP derivatives are effectively updated for enumerating local
minimum solutions. We theoretically analyze the proposed method and demonstrate
its usefulness through numerical experiments.
FastDepth: Fast Monocular Depth Estimation on Embedded Systems
Authors: Diana Wofk, Fangchang Ma, Tien-Ju Yang, Sertac Karaman, Vivienne Sze
Date: 2019-03-08 00:00:00
Tasks: CPU, Decoder, Depth Estimation, GPU, Monocular Depth Estimation, Network Pruning

Depth sensing is a critical function for robotic tasks such as localization,
mapping and obstacle detection. There has been a significant and growing
interest in depth estimation from a single RGB image, due to the relatively low
cost and size of monocular cameras. However, state-of-the-art single-view depth
estimation algorithms are based on fairly complex deep neural networks that are
too slow for real-time inference on an embedded platform, for instance, mounted
on a micro aerial vehicle. In this paper, we address the problem of fast depth
estimation on embedded systems. We propose an efficient and lightweight
encoder-decoder network architecture and apply network pruning to further
reduce computational complexity and latency. In particular, we focus on the
design of a low-latency decoder. Our methodology demonstrates that it is
possible to achieve similar accuracy as prior work on depth estimation, but at
inference speeds that are an order of magnitude faster. Our proposed network,
FastDepth, runs at 178 fps on an NVIDIA Jetson TX2 GPU and at 27 fps when using
only the TX2 CPU, with active power consumption under 10 W. FastDepth achieves
close to state-of-the-art accuracy on the NYU Depth v2 dataset. To the best of
the authors' knowledge, this paper demonstrates real-time monocular depth
estimation using a deep neural network with the lowest latency and highest
throughput on an embedded platform that can be carried by a micro aerial
vehicle.
A Deep Generative Model of Speech Complex Spectrograms
Authors: Aditya Arie Nugraha, Kouhei Sekiguchi, Kazuyoshi Yoshii
Date: 2019-03-08 00:00:00
Tasks: Decoder, model

This paper proposes an approach to the joint modeling of the short-time
Fourier transform magnitude and phase spectrograms with a deep generative
model. We assume that the magnitude follows a Gaussian distribution and the
phase follows a von Mises distribution. To improve the consistency of the phase
values in the time-frequency domain, we also apply the von Mises distribution
to the phase derivatives, i.e., the group delay and the instantaneous
frequency. Based on these assumptions, we explore and compare several
combinations of loss functions for training our models. Built upon the
variational autoencoder framework, our model consists of three convolutional
neural networks acting as an encoder, a magnitude decoder, and a phase decoder.
In addition to the latent variables, we propose to also condition the phase
estimation on the estimated magnitude. Evaluated for a time-domain speech
reconstruction task, our models could generate speech with a high perceptual
quality and a high intelligibility.
Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State
Authors: Richard Futrell, Ethan Wilcox, Takashi Morita, Peng Qian, Miguel Ballesteros, Roger Levy
Date: 2019-03-08 00:00:00
Proceeding: NAACL 2019 6

We deploy the methods of controlled psycholinguistic experimentation to shed
light on the extent to which the behavior of neural network language models
reflects incremental representations of syntactic state. To do so, we examine
model behavior on artificial sentences containing a variety of syntactically
complex structures. We test four models: two publicly available LSTM sequence
models of English (Jozefowicz et al., 2016; Gulordava et al., 2018) trained on
large datasets; an RNNG (Dyer et al., 2016) trained on a small, parsed dataset;
and an LSTM trained on the same small corpus as the RNNG. We find evidence that
the LSTMs trained on large datasets represent syntactic state over large spans
of text in a way that is comparable to the RNNG, while the LSTM trained on the
small dataset does not or does so only weakly.
Learn and Link: Learning Critical Regions for Efficient Planning
Authors: Daniel Molina, Kislay Kumar, Siddharth Srivastava
Date: 2019-03-08 00:00:00
Tasks: Motion Planning

This paper presents a new approach to learning for motion planning (MP) where critical regions of an environment are learned from a given set of motion plans and used to improve performance on new environments and problem instances. We introduce a new suite of sampling-based motion planners, Learn and Link. Our planners leverage critical regions to overcome the limitations of uniform sampling, while still maintaining guarantees of correctness inherent to sampling-based algorithms. We also show that convolutional neural networks (CNNs) can be used to identify critical regions for motion planning problems. We evaluate Learn and Link against planners from the Open Motion Planning Library (OMPL) using an extensive suite of experiments on challenging motion planning problems. We show that our approach requires far less planning time than existing sampling-based planners.
An Algorithm for the Visualization of Relevant Patterns in Astronomical Light Curves
Authors: Christian Pieringer, Karim Pichara, Márcio Catelán, Pavlos Protopapas
Date: 2019-03-08 00:00:00
Tasks: BIG-bench Machine Learning, Classification Of Variable Stars, Image Reconstruction, Time Series, Time Series Analysis

Within the last years, the classification of variable stars with Machine
Learning has become a mainstream area of research. Recently, visualization of
time series is attracting more attention in data science as a tool to visually
help scientists to recognize significant patterns in complex dynamics. Within
the Machine Learning literature, dictionary-based methods have been widely used
to encode relevant parts of image data. These methods intrinsically assign a
degree of importance to patches in pictures, according to their contribution in
the image reconstruction. Inspired by dictionary-based techniques, we present
an approach that naturally provides the visualization of salient parts in
astronomical light curves, making the analogy between image patches and
relevant pieces in time series. Our approach encodes the most meaningful
patterns such that we can approximately reconstruct light curves by just using
the encoded information. We test our method in light curves from the OGLE-III
and StarLight databases. Our results show that the proposed model delivers an
automatic and intuitive visualization of relevant light curve parts, such as
local peaks and drops in magnitude.
General Convolutional Sparse Coding with Unknown Noise
Authors: Yaqing Wang, James T. Kwok, Lionel M. Ni
Date: 2019-03-08 00:00:00

Convolutional sparse coding (CSC) can learn representative shift-invariant
patterns from multiple kinds of data. However, existing CSC methods can only
model noises from Gaussian distribution, which is restrictive and unrealistic.
In this paper, we propose a general CSC model capable of dealing with
complicated unknown noise. The noise is now modeled by Gaussian mixture model,
which can approximate any continuous probability density function. We use the
expectation-maximization algorithm to solve the problem and design an efficient
method for the weighted CSC problem in maximization step. The crux is to speed
up the convolution in the frequency domain while keeping the other computation
involving weight matrix in the spatial domain. Besides, we simultaneously
update the dictionary and codes by nonconvex accelerated proximal gradient
algorithm without bringing in extra alternating loops. The resultant method
obtains comparable time and space complexity compared with existing CSC
methods. Extensive experiments on synthetic and real noisy biomedical data sets
validate that our method can model noise effectively and obtain high-quality
filters and representation.
Learning Feature Relevance Through Step Size Adaptation in Temporal-Difference Learning
Authors: Alex Kearney, Vivek Veeriah, Jaden Travnik, Patrick M. Pilarski, Richard S. Sutton
Date: 2019-03-08 00:00:00
Tasks: Meta-Learning, Representation Learning

There is a long history of using meta learning as representation learning,
specifically for determining the relevance of inputs. In this paper, we examine
an instance of meta-learning in which feature relevance is learned by adapting
step size parameters of stochastic gradient descent---building on a variety of
prior work in stochastic approximation, machine learning, and artificial neural
networks. In particular, we focus on stochastic meta-descent introduced in the
Incremental Delta-Bar-Delta (IDBD) algorithm for setting individual step sizes
for each feature of a linear function approximator. Using IDBD, a feature with
large or small step sizes will have a large or small impact on generalization
from training examples. As a main contribution of this work, we extend IDBD to
temporal-difference (TD) learning---a form of learning which is effective in
sequential, non i.i.d. problems. We derive a variety of IDBD generalizations
for TD learning, demonstrating that they are able to distinguish which features
are relevant and which are not. We demonstrate that TD IDBD is effective at
learning feature relevance in both an idealized gridworld and a real-world
robotic prediction task.
Context-Aware Cross-Lingual Mapping
Authors: Hanan Aldarmaki, Mona Diab
Date: 2019-03-08 00:00:00
Proceeding: NAACL 2019 6
Tasks: Retrieval, Sentence, Sentence Embeddings, Translation, Word Embeddings, Word Translation

Cross-lingual word vectors are typically obtained by fitting an orthogonal
matrix that maps the entries of a bilingual dictionary from a source to a
target vector space. Word vectors, however, are most commonly used for sentence
or document-level representations that are calculated as the weighted average
of word embeddings. In this paper, we propose an alternative to word-level
mapping that better reflects sentence-level cross-lingual similarity. We
incorporate context in the transformation matrix by directly mapping the
averaged embeddings of aligned sentences in a parallel corpus. We also
implement cross-lingual mapping of deep contextualized word embeddings using
parallel sentences with word alignments. In our experiments, both approaches
resulted in cross-lingual sentence embeddings that outperformed
context-independent word mapping in sentence translation retrieval.
Furthermore, the sentence-level transformation could be used for word-level
mapping without loss in word translation quality.
Ranked List Loss for Deep Metric Learning
Authors: Xinshao Wang, Yang Hua, Elyor Kodirov, Neil M. Robertson
Date: 2019-03-08 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Retrieval, Metric Learning, Retrieval, Semantic Similarity, Semantic Textual Similarity

The objective of deep metric learning (DML) is to learn embeddings that can capture semantic similarity and dissimilarity information among data points. Existing pairwise or tripletwise loss functions used in DML are known to suffer from slow convergence due to a large proportion of trivial pairs or triplets as the model improves. To improve this, ranking-motivated structured losses are proposed recently to incorporate multiple examples and exploit the structured information among them. They converge faster and achieve state-of-the-art performance. In this work, we unveil two limitations of existing ranking-motivated structured losses and propose a novel ranked list loss to solve both of them. First, given a query, only a fraction of data points is incorporated to build the similarity structure. Consequently, some useful examples are ignored and the structure is less informative. To address this, we propose to build a set-based similarity structure by exploiting all instances in the gallery. The learning setting can be interpreted as few-shot retrieval: given a mini-batch, every example is iteratively used as a query, and the rest ones compose the gallery to search, i.e., the support set in few-shot setting. The rest examples are split into a positive set and a negative set. For every mini-batch, the learning objective of ranked list loss is to make the query closer to the positive set than to the negative set by a margin. Second, previous methods aim to pull positive pairs as close as possible in the embedding space. As a result, the intraclass data distribution tends to be extremely compressed. In contrast, we propose to learn a hypersphere for each class in order to preserve useful similarity structure inside it, which functions as regularisation. Extensive experiments demonstrate the superiority of our proposal by comparing with the state-of-the-art methods.
Fast Multichannel Source Separation Based on Jointly Diagonalizable Spatial Covariance Matrices
Authors: Kouhei Sekiguchi, Aditya Arie Nugraha, Yoshiaki Bando, Kazuyoshi Yoshii
Date: 2019-03-08 00:00:00
Proceeding: European Association for Signal Processing (EUSIPCO) 2019 9
Tasks: Speech Enhancement

This paper describes a versatile method that accelerates multichannel source
separation methods based on full-rank spatial modeling. A popular approach to
multichannel source separation is to integrate a spatial model with a source
model for estimating the spatial covariance matrices (SCMs) and power spectral
densities (PSDs) of each sound source in the time-frequency domain. One of the
most successful examples of this approach is multichannel nonnegative matrix
factorization (MNMF) based on a full-rank spatial model and a low-rank source
model. MNMF, however, is computationally expensive and often works poorly due
to the difficulty of estimating the unconstrained full-rank SCMs. Instead of
restricting the SCMs to rank-1 matrices with the severe loss of the spatial
modeling ability as in independent low-rank matrix analysis (ILRMA), we
restrict the SCMs of each frequency bin to jointly-diagonalizable but still
full-rank matrices. For such a fast version of MNMF, we propose a
computationally-efficient and convergence-guaranteed algorithm that is similar
in form to that of ILRMA. Similarly, we propose a fast version of a
state-of-the-art speech enhancement method based on a deep speech model and a
low-rank noise model. Experimental results showed that the fast versions of
MNMF and the deep speech enhancement method were several times faster and
performed even better than the original versions of those methods,
respectively.
Dyna-AIL : Adversarial Imitation Learning by Planning
Authors: Vaibhav Saxena, Srinivasan Sivanandan, Pulkit Mathur
Date: 2019-03-08 00:00:00
Tasks: Imitation Learning

Adversarial methods for imitation learning have been shown to perform well on
various control tasks. However, they require a large number of environment
interactions for convergence. In this paper, we propose an end-to-end
differentiable adversarial imitation learning algorithm in a Dyna-like
framework for switching between model-based planning and model-free learning
from expert data. Our results on both discrete and continuous environments show
that our approach of using model-based planning along with model-free learning
converges to an optimal policy with fewer number of environment interactions in
comparison to the state-of-the-art learning methods.
SeizureNet: Multi-Spectral Deep Feature Learning for Seizure Type Classification
Authors: Umar Asif, Subhrajit Roy, Jianbin Tang, Stefan Harrer
Date: 2019-03-08 00:00:00
Tasks: Classification, EEG, Electroencephalogram (EEG), General Classification, Knowledge Distillation, Management, Seizure Detection, Vocal Bursts Type Prediction

Automatic classification of epileptic seizure types in electroencephalograms (EEGs) data can enable more precise diagnosis and efficient management of the disease. This task is challenging due to factors such as low signal-to-noise ratios, signal artefacts, high variance in seizure semiology among epileptic patients, and limited availability of clinical data. To overcome these challenges, in this paper, we present SeizureNet, a deep learning framework which learns multi-spectral feature embeddings using an ensemble architecture for cross-patient seizure type classification. We used the recently released TUH EEG Seizure Corpus (V1.4.0 and V1.5.2) to evaluate the performance of SeizureNet. Experiments show that SeizureNet can reach a weighted F1 score of up to 0.94 for seizure-wise cross validation and 0.59 for patient-wise cross validation for scalp EEG based multi-class seizure type classification. We also show that the high-level feature embeddings learnt by SeizureNet considerably improve the accuracy of smaller networks through knowledge distillation for applications with low-memory constraints.
Pixel-Attentive Policy Gradient for Multi-Fingered Grasping in Cluttered Scenes
Authors: Bohan Wu, Iretiayo Akinola, Peter K. Allen
Date: 2019-03-08 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Robotic Grasping

Recent advances in on-policy reinforcement learning (RL) methods enabled learning agents in virtual environments to master complex tasks with high-dimensional and continuous observation and action spaces. However, leveraging this family of algorithms in multi-fingered robotic grasping remains a challenge due to large sim-to-real fidelity gaps and the high sample complexity of on-policy RL algorithms. This work aims to bridge these gaps by first reinforcement-learning a multi-fingered robotic grasping policy in simulation that operates in the pixel space of the input: a single depth image. Using a mapping from pixel space to Cartesian space according to the depth map, this method transfers to the real world with high fidelity and introduces a novel attention mechanism that substantially improves grasp success rate in cluttered environments. Finally, the direct-generative nature of this method allows learning of multi-fingered grasps that have flexible end-effector positions, orientations and rotations, as well as all degrees of freedom of the hand.
Pattern Recognition in SAR Images using Fractional Random Fields and its Possible Application to the Problem of the Detection of Oil Spills in Open Sea
Authors: Agustín Mailing, Segundo A. Molina, José L. Hamkalo, Fernando R. Dobarro, Juan M. Medina, Bruno Cernuschi-Frías, Daniel A. Fernández, Érica Schlaps
Date: 2019-03-07 00:00:00

In this note we deal with the detection of oil spills in open sea via self
similar, long range dependence random fields and wavelet filters. We show some
preliminary experimental results of our technique with Sentinel 1 SAR images.
Learning Hierarchical Teaching Policies for Cooperative Agents
Authors: Dong-Ki Kim, Miao Liu, Shayegan Omidshafiei, Sebastian Lopez-Cot, Matthew Riemer, Golnaz Habibi, Gerald Tesauro, Sami Mourad, Murray Campbell, Jonathan P. How
Date: 2019-03-07 00:00:00
Tasks: Transfer Learning

Collective learning can be greatly enhanced when agents effectively exchange knowledge with their peers. In particular, recent work studying agents that learn to teach other teammates has demonstrated that action advising accelerates team-wide learning. However, the prior work has simplified the learning of advising policies by using simple function approximations and only considered advising with primitive (low-level) actions, limiting the scalability of learning and teaching to complex domains. This paper introduces a novel learning-to-teach framework, called hierarchical multiagent teaching (HMAT), that improves scalability to complex environments by using the deep representation for student policies and by advising with more expressive extended action sequences over multiple levels of temporal abstraction. Our empirical evaluations demonstrate that HMAT improves team-wide learning progress in large, complex domains where previous approaches fail. HMAT also learns teaching policies that can effectively transfer knowledge to different teammates with knowledge of different tasks, even when the teammates have heterogeneous action spaces.
Unsupervised Domain Adaptation using Feature-Whitening and Consensus Loss
Authors: Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, Elisa Ricci
Date: 2019-03-07 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation, General Classification, Object Recognition, Unsupervised Domain Adaptation

A classifier trained on a dataset seldom works on other datasets obtained under different conditions due to domain shift. This problem is commonly addressed by domain adaptation methods. In this work we introduce a novel deep learning framework which unifies different paradigms in unsupervised domain adaptation. Specifically, we propose domain alignment layers which implement feature whitening for the purpose of matching source and target feature distributions. Additionally, we leverage the unlabeled target data by proposing the Min-Entropy Consensus loss, which regularizes training while avoiding the adoption of many user-defined hyper-parameters. We report results on publicly available datasets, considering both digit classification and object recognition tasks. We show that, in most of our experiments, our approach improves upon previous methods, setting new state-of-the-art performances.
Multi-Hot Compact Network Embedding
Authors: Chaozhuo Li, Senzhang Wang, Philip S. Yu, Zhoujun Li
Date: 2019-03-07 00:00:00
Tasks: Network Embedding, Representation Learning

Network embedding, as a promising way of the network representation learning, is capable of supporting various subsequent network mining and analysis tasks, and has attracted growing research interests recently. Traditional approaches assign each node with an independent continuous vector, which will cause huge memory overhead for large networks. In this paper we propose a novel multi-hot compact embedding strategy to effectively reduce memory cost by learning partially shared embeddings. The insight is that a node embedding vector is composed of several basis vectors, which can significantly reduce the number of continuous vectors while maintain similar data representation ability. Specifically, we propose a MCNE model to learn compact embeddings from pre-learned node features. A novel component named compressor is integrated into MCNE to tackle the challenge that popular back-propagation optimization cannot propagate through discrete samples. We further propose an end-to-end model MCNE$_{t}$ to learn compact embeddings from the input network directly. Empirically, we evaluate the proposed models over three real network datasets, and the results demonstrate that our proposals can save about 90\% of memory cost of network embeddings without significantly performance decline.