pwc-archive/papers-with-abstracts huggingface dataset:
Learning to Find Good Correspondences
Authors: Kwang Moo Yi, Eduard Trulls, Yuki Ono, Vincent Lepetit, Mathieu Salzmann, Pascal Fua
Date: 2017-11-16 00:00:00
Proceeding: CVPR 2018 6

We develop a deep architecture to learn to find good correspondences for
wide-baseline stereo. Given a set of putative sparse matches and the camera
intrinsics, we train our network in an end-to-end fashion to label the
correspondences as inliers or outliers, while simultaneously using them to
recover the relative pose, as encoded by the essential matrix. Our architecture
is based on a multi-layer perceptron operating on pixel coordinates rather than
directly on the image, and is thus simple and small. We introduce a novel
normalization technique, called Context Normalization, which allows us to
process each data point separately while imbuing it with global information,
and also makes the network invariant to the order of the correspondences. Our
experiments on multiple challenging datasets demonstrate that our method is
able to drastically improve the state of the art with little training data.
Faster Neural Network Training with Approximate Tensor Operations
Authors: Menachem Adelman, Kfir Y. Levy, Ido Hakimi, Mark Silberstein
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2021 12

We propose a novel technique for faster deep neural network training which systematically applies sample-based approximation to the constituent tensor operations, i.e., matrix multiplications and convolutions. We introduce new sampling techniques, study their theoretical properties, and prove that they provide the same convergence guarantees when applied to SGD training. We apply approximate tensor operations to single and multi-node training of MLP and CNN networks on MNIST, CIFAR-10 and ImageNet datasets. We demonstrate up to 66% reduction in the amount of computations and communication, and up to 1.37x faster training time while maintaining negligible or no impact on the final test accuracy.
Benchmarking Decoupled Neural Interfaces with Synthetic Gradients
Authors: Ekaba Bisong
Date: 2017-12-22 00:00:00
Tasks: Benchmarking

Artifical Neural Networks are a particular class of learning systems modeled
after biological neural functions with an interesting penchant for Hebbian
learning, that is "neurons that wire together, fire together". However, unlike
their natural counterparts, artificial neural networks have a close and
stringent coupling between the modules of neurons in the network. This coupling
or locking imposes upon the network a strict and inflexible structure that
prevent layers in the network from updating their weights until a full
feed-forward and backward pass has occurred. Such a constraint though may have
sufficed for a while, is now no longer feasible in the era of very-large-scale
machine learning, coupled with the increased desire for parallelization of the
learning process across multiple computing infrastructures. To solve this
problem, synthetic gradients (SG) with decoupled neural interfaces (DNI) are
introduced as a viable alternative to the backpropagation algorithm. This paper
performs a speed benchmark to compare the speed and accuracy capabilities of
SG-DNI as opposed to a standard neural interface using multilayer perceptron
MLP. SG-DNI shows good promise, in that it not only captures the learning
problem, it is also over 3-fold faster due to it asynchronous learning
capabilities.
NEWMA: a new method for scalable model-free online change-point detection
Authors: Nicolas Keriven, Damien Garreau, Iacopo Poli
Date: 2018-05-21 00:00:00
Tasks: Change Point Detection, Time Series, Time Series Analysis

We consider the problem of detecting abrupt changes in the distribution of a multi-dimensional time series, with limited computing power and memory. In this paper, we propose a new, simple method for model-free online change-point detection that relies only on fast and light recursive statistics, inspired by the classical Exponential Weighted Moving Average algorithm (EWMA). The proposed idea is to compute two EWMA statistics on the stream of data with different forgetting factors, and to compare them. By doing so, we show that we implicitly compare recent samples with older ones, without the need to explicitly store them. Additionally, we leverage Random Features (RFs) to efficiently use the Maximum Mean Discrepancy as a distance between distributions, furthermore exploiting recent optical hardware to compute high-dimensional RFs in near constant time. We show that our method is significantly faster than usual non-parametric methods for a given accuracy.
Channel Estimation for Visible Light Communications Using Neural Networks
Authors: Anil Yesilkaya, Onur Karatalay, Arif Selcuk Ogrenci, Erdal Panayirci
Date: 2018-05-21 00:00:00
Tasks: Prediction

Visible light communications (VLC) is an emerging field in technology and
research. Estimating the channel taps is a major requirement for designing
reliable communication systems. Due to the nonlinear characteristics of the VLC
channel those parameters cannot be derived easily. They can be calculated by
means of software simulation. In this work, a novel methodology is proposed for
the prediction of channel parameters using neural networks. Measurements
conducted in a controlled experimental setup are used to train neural networks
for channel tap prediction. Our experiment results indicate that neural
networks can be effectively trained to predict channel taps under different
environmental conditions.
Bayesian Inference of Spreading Processes on Networks
Authors: Ritabrata Dutta, Antonietta Mira, Jukka-Pekka Onnela
Date: 2017-09-26 00:00:00
Tasks: Bayesian Inference

Infectious diseases are studied to understand their spreading mechanisms, to
evaluate control strategies and to predict the risk and course of future
outbreaks. Because people only interact with a small number of individuals, and
because the structure of these interactions matters for spreading processes,
the pairwise relationships between individuals in a population can be usefully
represented by a network. Although the underlying processes of transmission are
different, the network approach can be used to study the spread of pathogens in
a contact network or the spread of rumors in an online social network. We study
simulated simple and complex epidemics on synthetic networks and on two
empirical networks, a social / contact network in an Indian village and an
online social network in the U.S. Our goal is to learn simultaneously about the
spreading process parameters and the source node (first infected node) of the
epidemic, given a fixed and known network structure, and observations about
state of nodes at several points in time. Our inference scheme is based on
approximate Bayesian computation (ABC), an inference technique for complex
models with likelihood functions that are either expensive to evaluate or
analytically intractable. ABC enables us to adopt a Bayesian approach to the
problem despite the posterior distribution being very complex. Our method is
agnostic about the topology of the network and the nature of the spreading
process. It generally performs well and, somewhat counter-intuitively, the
inference problem appears to be easier on more heterogeneous network
topologies, which enhances its future applicability to real-world settings
where few networks have homogeneous topologies.
Online Learning in Kernelized Markov Decision Processes
Authors: Sayak Ray Chowdhury, Aditya Gopalan
Date: 2018-05-21 00:00:00

We consider online learning for minimizing regret in unknown, episodic Markov
decision processes (MDPs) with continuous states and actions. We develop
variants of the UCRL and posterior sampling algorithms that employ
nonparametric Gaussian process priors to generalize across the state and action
spaces. When the transition and reward functions of the true MDP are members of
the associated Reproducing Kernel Hilbert Spaces of functions induced by
symmetric psd kernels (frequentist setting), we show that the algorithms enjoy
sublinear regret bounds. The bounds are in terms of explicit structural
parameters of the kernels, namely a novel generalization of the information
gain metric from kernelized bandit, and highlight the influence of transition
and reward function structure on the learning performance. Our results are
applicable to multidimensional state and action spaces with composite kernel
structures, and generalize results from the literature on kernelized bandits,
and the adaptive control of parametric linear dynamical systems with quadratic
costs.
Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labels
Authors: Bo Han, Quanming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, Masashi Sugiyama
Date: 2018-04-18 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Image Classification, Learning with noisy labels, Memorization

Deep learning with noisy labels is practically challenging, as the capacity
of deep models is so high that they can totally memorize these noisy labels
sooner or later during training. Nonetheless, recent studies on the
memorization effects of deep neural networks show that they would first
memorize training data of clean labels and then those of noisy labels.
Therefore in this paper, we propose a new deep learning paradigm called
Co-teaching for combating with noisy labels. Namely, we train two deep neural
networks simultaneously, and let them teach each other given every mini-batch:
firstly, each network feeds forward all data and selects some data of possibly
clean labels; secondly, two networks communicate with each other what data in
this mini-batch should be used for training; finally, each network back
propagates the data selected by its peer network and updates itself. Empirical
results on noisy versions of MNIST, CIFAR-10 and CIFAR-100 demonstrate that
Co-teaching is much superior to the state-of-the-art methods in the robustness
of trained deep models.
Exponentially Consistent Kernel Two-Sample Tests
Authors: Shengyu Zhu, Biao Chen, Zhitang Chen
Date: 2018-02-23 00:00:00
Tasks: Change Detection, Vocal Bursts Valence Prediction

Given two sets of independent samples from unknown distributions $P$ and $Q$,
a two-sample test decides whether to reject the null hypothesis that $P=Q$.
Recent attention has focused on kernel two-sample tests as the test statistics
are easy to compute, converge fast, and have low bias with their finite sample
estimates. However, there still lacks an exact characterization on the
asymptotic performance of such tests, and in particular, the rate at which the
type-II error probability decays to zero in the large sample limit. In this
work, we establish that a class of kernel two-sample tests are exponentially
consistent with Polish, locally compact Hausdorff sample space, e.g., $\mathbb
R^d$. The obtained exponential decay rate is further shown to be optimal among
all two-sample tests satisfying the level constraint, and is independent of
particular kernels provided that they are bounded continuous and
characteristic. Our results gain new insights into related issues such as fair
alternative for testing and kernel selection strategy. Finally, as an
application, we show that a kernel based test achieves the optimal detection
for off-line change detection in the nonparametric setting.
A Talker Ensemble: the University of Wrocław's Entry to the NIPS 2017 Conversational Intelligence Challenge
Authors: Jan Chorowski, Adrian Łańcucki, Szymon Malik, Maciej Pawlikowski, Paweł Rychlikowski, Paweł Zykowski
Date: 2018-05-21 00:00:00
Tasks: Chatbot, Question Answering

We present Poetwannabe, a chatbot submitted by the University of Wroc{\l}aw
to the NIPS 2017 Conversational Intelligence Challenge, in which it ranked
first ex-aequo. It is able to conduct a conversation with a user in a natural
language. The primary functionality of our dialogue system is context-aware
question answering (QA), while its secondary function is maintaining user
engagement. The chatbot is composed of a number of sub-modules, which
independently prepare replies to user's prompts and assess their own
confidence. To answer questions, our dialogue system relies heavily on factual
data, sourced mostly from Wikipedia and DBpedia, data of real user interactions
in public forums, as well as data concerning general literature. Where
applicable, modules are trained on large datasets using GPUs. However, to
comply with the competition's requirements, the final system is compact and
runs on commodity hardware.
Incorporating Glosses into Neural Word Sense Disambiguation
Authors: Fuli Luo, Tianyu Liu, Qiaolin Xia, Baobao Chang, Zhifang Sui
Date: 2018-05-21 00:00:00
Proceeding: ACL 2018 7
Tasks: Word Sense Disambiguation

Word Sense Disambiguation (WSD) aims to identify the correct meaning of
polysemous words in the particular context. Lexical resources like WordNet
which are proved to be of great help for WSD in the knowledge-based methods.
However, previous neural networks for WSD always rely on massive labeled data
(context), ignoring lexical resources like glosses (sense definitions). In this
paper, we integrate the context and glosses of the target word into a unified
framework in order to make full use of both labeled data and lexical knowledge.
Therefore, we propose GAS: a gloss-augmented WSD neural network which jointly
encodes the context and glosses of the target word. GAS models the semantic
relationship between the context and the gloss in an improved memory network
framework, which breaks the barriers of the previous supervised methods and
knowledge-based methods. We further extend the original gloss of word sense via
its semantic relations in WordNet to enrich the gloss information. The
experimental results show that our model outperforms the state-of-theart
systems on several English all-words WSD datasets.
DiDA: Disentangled Synthesis for Domain Adaptation
Authors: Jinming Cao, Oren Katzir, Peng Jiang, Dani Lischinski, Danny Cohen-Or, Changhe Tu, Yangyan Li
Date: 2018-05-21 00:00:00
Tasks: Disentanglement, Domain Adaptation, Unsupervised Domain Adaptation

Unsupervised domain adaptation aims at learning a shared model for two
related, but not identical, domains by leveraging supervision from a source
domain to an unsupervised target domain. A number of effective domain
adaptation approaches rely on the ability to extract discriminative, yet
domain-invariant, latent factors which are common to both domains. Extracting
latent commonality is also useful for disentanglement analysis, enabling
separation between the common and the domain-specific features of both domains.
In this paper, we present a method for boosting domain adaptation performance
by leveraging disentanglement analysis. The key idea is that by learning to
separately extract both the common and the domain-specific features, one can
synthesize more target domain data with supervision, thereby boosting the
domain adaptation performance. Better common feature extraction, in turn, helps
further improve the disentanglement analysis and disentangled synthesis. We
show that iterating between domain adaptation and disentanglement analysis can
consistently improve each other on several unsupervised domain adaptation
tasks, for various domain adaptation backbone models.
Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behavior
Authors: Siddharth Reddy, Anca D. Dragan, Sergey Levine
Date: 2018-05-21 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Inferring intent from observed behavior has been studied extensively within
the frameworks of Bayesian inverse planning and inverse reinforcement learning.
These methods infer a goal or reward function that best explains the actions of
the observed agent, typically a human demonstrator. Another agent can use this
inferred intent to predict, imitate, or assist the human user. However, a
central assumption in inverse reinforcement learning is that the demonstrator
is close to optimal. While models of suboptimal behavior exist, they typically
assume that suboptimal actions are the result of some type of random noise or a
known cognitive bias, like temporal inconsistency. In this paper, we take an
alternative approach, and model suboptimal behavior as the result of internal
model misspecification: the reason that user actions might deviate from
near-optimal actions is that the user has an incorrect set of beliefs about the
rules -- the dynamics -- governing how actions affect the environment. Our
insight is that while demonstrated actions may be suboptimal in the real world,
they may actually be near-optimal with respect to the user's internal model of
the dynamics. By estimating these internal beliefs from observed behavior, we
arrive at a new method for inferring intent. We demonstrate in simulation and
in a user study with 12 participants that this approach enables us to more
accurately model human intent, and can be used in a variety of applications,
including offering assistance in a shared autonomy framework and inferring
human preferences.
Object Detection in Equirectangular Panorama
Authors: Wenyan Yang, Yanlin Qian, Francesco Cricri, Lixin Fan, Joni-Kristian Kamarainen
Date: 2018-05-21 00:00:00
Tasks: GPU, Object, object-detection, Object Detection

We introduced a high-resolution equirectangular panorama (360-degree, virtual
reality) dataset for object detection and propose a multi-projection variant of
YOLO detector. The main challenge with equirectangular panorama image are i)
the lack of annotated training data, ii) high-resolution imagery and iii)
severe geometric distortions of objects near the panorama projection poles. In
this work, we solve the challenges by i) using training examples available in
the "conventional datasets" (ImageNet and COCO), ii) employing only
low-resolution images that require only moderate GPU computing power and
memory, and iii) our multi-projection YOLO handles projection distortions by
making multiple stereographic sub-projections. In our experiments, YOLO
outperforms the other state-of-art detector, Faster RCNN and our
multi-projection YOLO achieves the best accuracy with low-resolution input.
Bidirectional Learning for Robust Neural Networks
Authors: Sidney Pontes-Filho, Marcus Liwicki
Date: 2018-05-21 00:00:00

A multilayer perceptron can behave as a generative classifier by applying
bidirectional learning (BL). It consists of training an undirected neural
network to map input to output and vice-versa; therefore it can produce a
classifier in one direction, and a generator in the opposite direction for the
same data. The learning process of BL tries to reproduce the neuroplasticity
stated in Hebbian theory using only backward propagation of errors. In this
paper, two novel learning techniques are introduced which use BL for improving
robustness to white noise static and adversarial examples. The first method is
bidirectional propagation of errors, which the error propagation occurs in
backward and forward directions. Motivated by the fact that its generative
model receives as input a constant vector per class, we introduce as a second
method the hybrid adversarial networks (HAN). Its generative model receives a
random vector as input and its training is based on generative adversarial
networks (GAN). To assess the performance of BL, we perform experiments using
several architectures with fully and convolutional layers, with and without
bias. Experimental results show that both methods improve robustness to white
noise static and adversarial examples, and even increase accuracy, but have
different behavior depending on the architecture and task, being more
beneficial to use the one or the other. Nevertheless, HAN using a convolutional
architecture with batch normalization presents outstanding robustness, reaching
state-of-the-art accuracy on adversarial examples of hand-written digits.
Adversarial Noise Layer: Regularize Neural Network By Adding Noise
Authors: Zhonghui You, Jinmian Ye, Kunming Li, Zenglin Xu, Ping Wang
Date: 2018-05-21 00:00:00

In this paper, we introduce a novel regularization method called Adversarial
Noise Layer (ANL) and its efficient version called Class Adversarial Noise
Layer (CANL), which are able to significantly improve CNN's generalization
ability by adding carefully crafted noise into the intermediate layer
activations. ANL and CANL can be easily implemented and integrated with most of
the mainstream CNN-based models. We compared the effects of the different types
of noise and visually demonstrate that our proposed adversarial noise instruct
CNN models to learn to extract cleaner feature maps, which further reduce the
risk of over-fitting. We also conclude that models trained with ANL or CANL are
more robust to the adversarial examples generated by FGSM than the traditional
adversarial training approaches.
Exploration on Generating Traditional Chinese Medicine Prescription from Symptoms with an End-to-End method
Authors: Wei Li, Zheng Yang, Xu sun
Date: 2018-01-27 00:00:00
Tasks: Decoder

Traditional Chinese Medicine (TCM) is an influential form of medical
treatment in China and surrounding areas. In this paper, we propose a TCM
prescription generation task that aims to automatically generate a herbal
medicine prescription based on textual symptom descriptions.
Sequence-to-sequence (seq2seq) model has been successful in dealing with
sequence generation tasks. We explore a potential end-to-end solution to the
TCM prescription generation task using seq2seq models. However, experiments
show that directly applying seq2seq model leads to unfruitful results due to
the repetition problem. To solve the problem, we propose a novel decoder with
coverage mechanism and a novel soft loss function. The experimental results
demonstrate the effectiveness of the proposed approach. Judged by professors
who excel in TCM, the generated prescriptions are rated 7.3 out of 10. It shows
that the model can indeed help with the prescribing procedure in real life.
Global Minimum for a Finsler Elastica Minimal Path Approach
Authors: Da Chen, Jean-Marie Mirebeau, Laurent D. Cohen
Date: 2016-12-01 00:00:00
Tasks: Contour Detection

In this paper, we propose a novel curvature-penalized minimal path model via
an orientation-lifted Finsler metric and the Euler elastica curve. The original
minimal path model computes the globally minimal geodesic by solving an Eikonal
partial differential equation (PDE). Essentially, this first-order model is
unable to penalize curvature which is related to the path rigidity property in
the classical active contour models. To solve this problem, we present an
Eikonal PDE-based Finsler elastica minimal path approach to address the
curvature-penalized geodesic energy minimization problem. We were successful at
adding the curvature penalization to the classical geodesic energy. The basic
idea of this work is to interpret the Euler elastica bending energy via a novel
Finsler elastica metric that embeds a curvature penalty. This metric is
non-Riemannian, anisotropic and asymmetric, and is defined over an
orientation-lifted space by adding to the image domain the orientation as an
extra space dimension. Based on this orientation lifting, the proposed minimal
path model can benefit from both the curvature and orientation of the paths.
Thanks to the fast marching method, the global minimum of the
curvature-penalized geodesic energy can be computed efficiently. We introduce
two anisotropic image data-driven speed functions that are computed by
steerable filters. Based on these orientation-dependent speed functions, we can
apply the proposed Finsler elastica minimal path model to the applications of
closed contour detection, perceptual grouping and tubular structure extraction.
Numerical experiments on both synthetic and real images show that these
applications of the proposed model indeed obtain promising results.