pwc-archive/papers-with-abstracts huggingface dataset:
Multitask Learning with CTC and Segmental CRF for Speech Recognition
Authors: Liang Lu, Lingpeng Kong, Chris Dyer, Noah A. Smith
Date: 2017-02-21 00:00:00
Tasks: speech-recognition, Speech Recognition

Segmental conditional random fields (SCRFs) and connectionist temporal
classification (CTC) are two sequence labeling methods used for end-to-end
training of speech recognition models. Both models define a transcription
probability by marginalizing decisions about latent segmentation alternatives
to derive a sequence probability: the former uses a globally normalized joint
model of segment labels and durations, and the latter classifies each frame as
either an output symbol or a "continuation" of the previous label. In this
paper, we train a recognition model by optimizing an interpolation between the
SCRF and CTC losses, where the same recurrent neural network (RNN) encoder is
used for feature extraction for both outputs. We find that this multitask
objective improves recognition accuracy when decoding with either the SCRF or
CTC models. Additionally, we show that CTC can also be used to pretrain the RNN
encoder, which improves the convergence rate when learning the joint model.
A Joint Model for Question Answering and Question Generation
Authors: Tong Wang, Xingdi Yuan, Adam Trischler
Date: 2017-06-05 00:00:00
Tasks: Question Answering, Question Generation, Question-Generation, Reading Comprehension

We propose a generative machine comprehension model that learns jointly to
ask and answer questions based on documents. The proposed model uses a
sequence-to-sequence framework that encodes the document and generates a
question (answer) given an answer (question). Significant improvement in model
performance is observed empirically on the SQuAD corpus, confirming our
hypothesis that the model benefits from jointly learning to perform both tasks.
We believe the joint model's novelty offers a new perspective on machine
comprehension beyond architectural engineering, and serves as a first step
towards autonomous information seeking.
Types of Cognition and its Implications for future High-Level Cognitive Machines
Authors: Camilo Miguel Signorelli
Date: 2017-06-05 00:00:00
Tasks: Attribute, Relation, Vocal Bursts Intensity Prediction

This work summarizes part of current knowledge on High-level Cognitive
process and its relation with biological hardware. Thus, it is possible to
identify some paradoxes which could impact the development of future
technologies and artificial intelligence: we may make a High-level Cognitive
Machine, sacrificing the principal attribute of a machine, its accuracy.
Visual Interaction Networks
Authors: Nicholas Watters, Andrea Tacchetti, Theophane Weber, Razvan Pascanu, Peter Battaglia, Daniel Zoran
Date: 2017-06-05 00:00:00
Tasks: Decision Making

From just a glance, humans can make rich predictions about the future state
of a wide range of physical systems. On the other hand, modern approaches from
engineering, robotics, and graphics are often restricted to narrow domains and
require direct measurements of the underlying states. We introduce the Visual
Interaction Network, a general-purpose model for learning the dynamics of a
physical system from raw visual observations. Our model consists of a
perceptual front-end based on convolutional neural networks and a dynamics
predictor based on interaction networks. Through joint training, the perceptual
front-end learns to parse a dynamic visual scene into a set of factored latent
object representations. The dynamics predictor learns to roll these states
forward in time by computing their interactions and dynamics, producing a
predicted physical trajectory of arbitrary length. We found that from just six
input video frames the Visual Interaction Network can generate accurate future
trajectories of hundreds of time steps on a wide range of physical systems. Our
model can also be applied to scenes with invisible objects, inferring their
future states from their effects on the visible objects, and can implicitly
infer the unknown mass of objects. Our results demonstrate that the perceptual
module and the object-based dynamics predictor module can induce factored
latent representations that support accurate dynamical predictions. This work
opens new opportunities for model-based decision-making and planning from raw
sensory observations in complex physical environments.
A simple neural network module for relational reasoning
Authors: Adam Santoro, David Raposo, David G. T. Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, Timothy Lillicrap
Date: 2017-06-05 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Image Retrieval with Multi-Modal Query, Question Answering, Relational Reasoning, Visual Question Answering, Visual Question Answering (VQA)

Relational reasoning is a central component of generally intelligent
behavior, but has proven difficult for neural networks to learn. In this paper
we describe how to use Relation Networks (RNs) as a simple plug-and-play module
to solve problems that fundamentally hinge on relational reasoning. We tested
RN-augmented networks on three tasks: visual question answering using a
challenging dataset called CLEVR, on which we achieve state-of-the-art,
super-human performance; text-based question answering using the bAbI suite of
tasks; and complex reasoning about dynamic physical systems. Then, using a
curated dataset called Sort-of-CLEVR we show that powerful convolutional
networks do not have a general capacity to solve relational questions, but can
gain this capacity when augmented with RNs. Our work shows how a deep learning
architecture equipped with an RN module can implicitly discover and learn to
reason about entities and their relations.
Learning Whenever Learning is Possible: Universal Learning under General Stochastic Processes
Authors: Steve Hanneke
Date: 2017-06-05 00:00:00
Tasks: Learning Theory

This work initiates a general study of learning and generalization without the i.i.d. assumption, starting from first principles. While the traditional approach to statistical learning theory typically relies on standard assumptions from probability theory (e.g., i.i.d. or stationary ergodic), in this work we are interested in developing a theory of learning based only on the most fundamental and necessary assumptions implicit in the requirements of the learning problem itself. We specifically study universally consistent function learning, where the objective is to obtain low long-run average loss for any target function, when the data follow a given stochastic process. We are then interested in the question of whether there exist learning rules guaranteed to be universally consistent given only the assumption that universally consistent learning is possible for the given data process. The reasoning that motivates this criterion emanates from a kind of optimist's decision theory, and so we refer to such learning rules as being optimistically universal. We study this question in three natural learning settings: inductive, self-adaptive, and online. Remarkably, as our strongest positive result, we find that optimistically universal learning rules do indeed exist in the self-adaptive learning setting. Establishing this fact requires us to develop new approaches to the design of learning algorithms. Along the way, we also identify concise characterizations of the family of processes under which universally consistent learning is possible in the inductive and self-adaptive settings. We additionally pose a number of enticing open problems, particularly for the online learning setting.
A method for the online construction of the set of states of a Markov Decision Process using Answer Set Programming
Authors: Leonardo A. Ferreira, Reinaldo A. C. Bianchi, Paulo E. Santos, Ramon Lopez de Mantaras
Date: 2017-06-05 00:00:00
Tasks: Decision Making, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Non-stationary domains, that change in unpredicted ways, are a challenge for
agents searching for optimal policies in sequential decision-making problems.
This paper presents a combination of Markov Decision Processes (MDP) with
Answer Set Programming (ASP), named {\em Online ASP for MDP} (oASP(MDP)), which
is a method capable of constructing the set of domain states while the agent
interacts with a changing environment. oASP(MDP) updates previously obtained
policies, learnt by means of Reinforcement Learning (RL), using rules that
represent the domain changes observed by the agent. These rules represent a set
of domain constraints that are processed as ASP programs reducing the search
space. Results show that oASP(MDP) is capable of finding solutions for problems
in non-stationary domains without interfering with the action-value function
approximation process.
Teaching Machines to Describe Images via Natural Language Feedback
Authors: Huan Ling, Sanja Fidler
Date: 2017-06-01 00:00:00
Tasks: Descriptive, Image Captioning, Sentence

Robots will eventually be part of every household. It is thus critical to
enable algorithms to learn from and be guided by non-expert users. In this
paper, we bring a human in the loop, and enable a human teacher to give
feedback to a learning agent in the form of natural language. We argue that a
descriptive sentence can provide a much stronger learning signal than a numeric
reward in that it can easily point to where the mistakes are and how to correct
them. We focus on the problem of image captioning in which the quality of the
output can easily be judged by non-experts. We propose a hierarchical
phrase-based captioning model trained with policy gradients, and design a
feedback network that provides reward to the learner by conditioning on the
human-provided feedback. We show that by exploiting descriptive feedback our
model learns to perform better than when given independently written human
captions.
Multi-Observation Elicitation
Authors: Sebastian Casalaina-Martin, Rafael Frongillo, Tom Morgan, Bo Waggoner
Date: 2017-06-05 00:00:00
Tasks: BIG-bench Machine Learning

We study loss functions that measure the accuracy of a prediction based on
multiple data points simultaneously. To our knowledge, such loss functions have
not been studied before in the area of property elicitation or in machine
learning more broadly. As compared to traditional loss functions that take only
a single data point, these multi-observation loss functions can in some cases
drastically reduce the dimensionality of the hypothesis required. In
elicitation, this corresponds to requiring many fewer reports; in empirical
risk minimization, it corresponds to algorithms on a hypothesis space of much
smaller dimension. We explore some examples of the tradeoff between
dimensionality and number of observations, give some geometric
characterizations and intuition for relating loss functions and the properties
that they elicit, and discuss some implications for both elicitation and
machine-learning contexts.
Sparse Stochastic Bandits
Authors: Joon Kwon, Vianney Perchet, Claire Vernade
Date: 2017-06-05 00:00:00

In the classical multi-armed bandit problem, d arms are available to the
decision maker who pulls them sequentially in order to maximize his cumulative
reward. Guarantees can be obtained on a relative quantity called regret, which
scales linearly with d (or with sqrt(d) in the minimax sense). We here consider
the sparse case of this classical problem in the sense that only a small number
of arms, namely s < d, have a positive expected reward. We are able to leverage
this additional assumption to provide an algorithm whose regret scales with s
instead of d. Moreover, we prove that this algorithm is optimal by providing a
matching lower bound - at least for a wide and pertinent range of parameters
that we determine - and by evaluating its performance on simulated data.
Calibrating Black Box Classification Models through the Thresholding Method
Authors: Arun Srinivasan
Date: 2017-05-20 00:00:00
Tasks: Classification, General Classification

In high-dimensional classification settings, we wish to seek a balance
between high power and ensuring control over a desired loss function. In many
settings, the points most likely to be misclassified are those who lie near the
decision boundary of the given classification method. Often, these
uninformative points should not be classified as they are noisy and do not
exhibit strong signals. In this paper, we introduce the Thresholding Method to
parameterize the problem of determining which points exhibit strong signals and
should be classified. We demonstrate the empirical performance of this novel
calibration method in providing loss function control at a desired level, as
well as explore how the method assuages the effect of overfitting. We explore
the benefits of error control through the Thresholding Method in difficult,
high-dimensional, simulated settings. Finally, we show the flexibility of the
Thresholding Method through applying the method in a variety of real data
settings.
The Geometry of Nodal Sets and Outlier Detection
Authors: Xiuyuan Cheng, Gal Mishne, Stefan Steinerberger
Date: 2017-06-05 00:00:00
Tasks: Outlier Detection

Let $(M,g)$ be a compact manifold and let $-\Delta \phi_k = \lambda_k \phi_k$
be the sequence of Laplacian eigenfunctions. We present a curious new
phenomenon which, so far, we only managed to understand in a few highly
specialized cases: the family of functions $f_N:M \rightarrow \mathbb{R}_{\geq
0}$ $$ f_N(x) = \sum_{k \leq N}{ \frac{1}{\sqrt{\lambda_k}}
\frac{|\phi_k(x)|}{\|\phi_k\|_{L^{\infty}(M)}}}$$ seems strangely suited for
the detection of anomalous points on the manifold. It may be heuristically
interpreted as the sum over distances to the nearest nodal line and potentially
hints at a new phenomenon in spectral geometry. We give rigorous statements on
the unit square $[0,1]^2$ (where minima localize in $\mathbb{Q}^2$) and on
Paley graphs (where $f_N$ recovers the geometry of quadratic residues of the
underlying finite field $\mathbb{F}_p$). Numerical examples show that the
phenomenon seems to arise on fairly generic manifolds.
Neuroevolution on the Edge of Chaos
Authors: Filip Matzner
Date: 2017-06-05 00:00:00

Echo state networks represent a special type of recurrent neural networks.
Recent papers stated that the echo state networks maximize their computational
performance on the transition between order and chaos, the so-called edge of
chaos. This work confirms this statement in a comprehensive set of experiments.
Furthermore, the echo state networks are compared to networks evolved via
neuroevolution. The evolved networks outperform the echo state networks,
however, the evolution consumes significant computational resources. It is
demonstrated that echo state networks with local connections combine the best
of both worlds, the simplicity of random echo state networks and the
performance of evolved networks. Finally, it is shown that evolution tends to
stay close to the ordered side of the edge of chaos.
Neural Decomposition of Time-Series Data for Effective Generalization
Authors: Luke B. Godfrey, Michael S. Gashler
Date: 2017-05-25 00:00:00
Tasks: Time Series, Time Series Analysis, Time Series Forecasting

We present a neural network technique for the analysis and extrapolation of
time-series data called Neural Decomposition (ND). Units with a sinusoidal
activation function are used to perform a Fourier-like decomposition of
training samples into a sum of sinusoids, augmented by units with nonperiodic
activation functions to capture linear trends and other nonperiodic components.
We show how careful weight initialization can be combined with regularization
to form a simple model that generalizes well. Our method generalizes
effectively on the Mackey-Glass series, a dataset of unemployment rates as
reported by the U.S. Department of Labor Statistics, a time-series of monthly
international airline passengers, the monthly ozone concentration in downtown
Los Angeles, and an unevenly sampled time-series of oxygen isotope measurements
from a cave in north India. We find that ND outperforms popular time-series
forecasting techniques including LSTM, echo state networks, ARIMA, SARIMA, SVR
with a radial basis function, and Gashler and Ashmore's model.
Submanifold Sparse Convolutional Networks
Authors: Benjamin Graham, Laurens van der Maaten
Date: 2017-06-05 00:00:00
Tasks: 3D Part Segmentation

Convolutional network are the de-facto standard for analysing spatio-temporal
data such as images, videos, 3D shapes, etc. Whilst some of this data is
naturally dense (for instance, photos), many other data sources are inherently
sparse. Examples include pen-strokes forming on a piece of paper, or (colored)
3D point clouds that were obtained using a LiDAR scanner or RGB-D camera.
Standard "dense" implementations of convolutional networks are very inefficient
when applied on such sparse data. We introduce a sparse convolutional operation
tailored to processing sparse data that differs from prior work on sparse
convolutional networks in that it operates strictly on submanifolds, rather
than "dilating" the observation with every layer in the network. Our empirical
analysis of the resulting submanifold sparse convolutional networks shows that
they perform on par with state-of-the-art methods whilst requiring
substantially less computation.
Audio Scene Classification with Deep Recurrent Neural Networks
Authors: Huy Phan, Philipp Koch, Fabrice Katzberg, Marco Maass, Radoslaw Mazur, Alfred Mertins
Date: 2017-03-14 00:00:00
Tasks: Classification, General Classification, Scene Classification

We introduce in this work an efficient approach for audio scene
classification using deep recurrent neural networks. An audio scene is firstly
transformed into a sequence of high-level label tree embedding feature vectors.
The vector sequence is then divided into multiple subsequences on which a deep
GRU-based recurrent neural network is trained for sequence-to-label
classification. The global predicted label for the entire sequence is finally
obtained via aggregation of subsequence classification outputs. We will show
that our approach obtains an F1-score of 97.7% on the LITIS Rouen dataset,
which is the largest dataset publicly available for the task. Compared to the
best previously reported result on the dataset, our approach is able to reduce
the relative classification error by 35.3%.
Deep Feature Flow for Video Recognition
Authors: Xizhou Zhu, Yuwen Xiong, Jifeng Dai, Lu Yuan, Yichen Wei
Date: 2016-11-23 00:00:00
Proceeding: CVPR 2017 7
Tasks: Video Recognition, Video Semantic Segmentation

Deep convolutional neutral networks have achieved great success on image
recognition tasks. Yet, it is non-trivial to transfer the state-of-the-art
image recognition networks to videos as per-frame evaluation is too slow and
unaffordable. We present deep feature flow, a fast and accurate framework for
video recognition. It runs the expensive convolutional sub-network only on
sparse key frames and propagates their deep feature maps to other frames via a
flow field. It achieves significant speedup as flow computation is relatively
fast. The end-to-end training of the whole architecture significantly boosts
the recognition accuracy. Deep feature flow is flexible and general. It is
validated on two recent large scale video datasets. It makes a large step
towards practical video recognition.
Deformable Convolutional Networks
Authors: Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, Yichen Wei
Date: 2017-03-17 00:00:00
Proceeding: ICCV 2017 10
Tasks: Object Detection, Semantic Segmentation, Vessel Detection

Convolutional neural networks (CNNs) are inherently limited to model
geometric transformations due to the fixed geometric structures in its building
modules. In this work, we introduce two new modules to enhance the
transformation modeling capacity of CNNs, namely, deformable convolution and
deformable RoI pooling. Both are based on the idea of augmenting the spatial
sampling locations in the modules with additional offsets and learning the
offsets from target tasks, without additional supervision. The new modules can
readily replace their plain counterparts in existing CNNs and can be easily
trained end-to-end by standard back-propagation, giving rise to deformable
convolutional networks. Extensive experiments validate the effectiveness of our
approach on sophisticated vision tasks of object detection and semantic
segmentation. The code would be released.
Bayesian LSTMs in medicine
Authors: Jos van der Westhuizen, Joan Lasenby
Date: 2017-06-05 00:00:00
Tasks: BIG-bench Machine Learning, Deep Learning, General Classification, Time Series, Time Series Analysis

The medical field stands to see significant benefits from the recent advances
in deep learning. Knowing the uncertainty in the decision made by any machine
learning algorithm is of utmost importance for medical practitioners. This
study demonstrates the utility of using Bayesian LSTMs for classification of
medical time series. Four medical time series datasets are used to show the
accuracy improvement Bayesian LSTMs provide over standard LSTMs. Moreover, we
show cherry-picked examples of confident and uncertain classifications of the
medical time series. With simple modifications of the common practice for deep
learning, significant improvements can be made for the medical practitioner and
patient.
Learning Structured Semantic Embeddings for Visual Recognition
Authors: Dong Li, Hsin-Ying Lee, Jia-Bin Huang, Shengjin Wang, Ming-Hsuan Yang
Date: 2017-06-05 00:00:00
Tasks: General Classification, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION, Word Embeddings, Zero-Shot Learning

Numerous embedding models have been recently explored to incorporate semantic
knowledge into visual recognition. Existing methods typically focus on
minimizing the distance between the corresponding images and texts in the
embedding space but do not explicitly optimize the underlying structure. Our
key observation is that modeling the pairwise image-image relationship improves
the discrimination ability of the embedding model. In this paper, we propose
the structured discriminative and difference constraints to learn
visual-semantic embeddings. First, we exploit the discriminative constraints to
capture the intra- and inter-class relationships of image embeddings. The
discriminative constraints encourage separability for image instances of
different classes. Second, we align the difference vector between a pair of
image embeddings with that of the corresponding word embeddings. The difference
constraints help regularize image embeddings to preserve the semantic
relationships among word embeddings. Extensive evaluations demonstrate the
effectiveness of the proposed structured embeddings for single-label
classification, multi-label classification, and zero-shot recognition.
Boosted Multiple Kernel Learning for First-Person Activity Recognition
Authors: Fatih Ozkan, Mehmet Ali Arabaci, Elif Surer, Alptekin Temizel
Date: 2017-02-22 00:00:00
Tasks: Activity Recognition

Activity recognition from first-person (ego-centric) videos has recently
gained attention due to the increasing ubiquity of the wearable cameras. There
has been a surge of efforts adapting existing feature descriptors and designing
new descriptors for the first-person videos. An effective activity recognition
system requires selection and use of complementary features and appropriate
kernels for each feature. In this study, we propose a data-driven framework for
first-person activity recognition which effectively selects and combines
features and their respective kernels during the training. Our experimental
results show that use of Multiple Kernel Learning (MKL) and Boosted MKL in
first-person activity recognition problem exhibits improved results in
comparison to the state-of-the-art. In addition, these techniques enable the
expansion of the framework with new features in an efficient and convenient
way.