pwc-archive/papers-with-abstracts huggingface dataset:
Word learning and the acquisition of syntactic--semantic overhypotheses
Authors: Jon Gauthier, Roger Levy, Joshua B. Tenenbaum
Date: 2018-05-14 00:00:00
Tasks: Language Acquisition

Children learning their first language face multiple problems of induction:
how to learn the meanings of words, and how to build meaningful phrases from
those words according to syntactic rules. We consider how children might solve
these problems efficiently by solving them jointly, via a computational model
that learns the syntax and semantics of multi-word utterances in a grounded
reference game. We select a well-studied empirical case in which children are
aware of patterns linking the syntactic and semantic properties of words ---
that the properties picked out by base nouns tend to be related to shape, while
prenominal adjectives tend to refer to other properties such as color. We show
that children applying such inductive biases are accurately reflecting the
statistics of child-directed speech, and that inducing similar biases in our
computational model captures children's behavior in a classic adjective
learning experiment. Our model incorporating such biases also demonstrates a
clear data efficiency in learning, relative to a baseline model that learns
without forming syntax-sensitive overhypotheses of word meaning. Thus solving a
more complex joint inference problem may make the full problem of language
acquisition easier, not harder.
A comparable study of modeling units for end-to-end Mandarin speech recognition
Authors: Wei Zou, Dongwei Jiang, Shuaijiang Zhao, Xiangang Li
Date: 2018-05-10 00:00:00
Tasks: Decoder, speech-recognition, Speech Recognition

End-To-End speech recognition have become increasingly popular in mandarin
speech recognition and achieved delightful performance.
  Mandarin is a tonal language which is different from English and requires
special treatment for the acoustic modeling units. There have been several
different kinds of modeling units for mandarin such as phoneme, syllable and
Chinese character.
  In this work, we explore two major end-to-end models: connectionist temporal
classification (CTC) model and attention based encoder-decoder model for
mandarin speech recognition. We compare the performance of three different
scaled modeling units: context dependent phoneme(CDP), syllable with tone and
Chinese character.
  We find that all types of modeling units can achieve approximate character
error rate (CER) in CTC model and the performance of Chinese character
attention model is better than syllable attention model. Furthermore, we find
that Chinese character is a reasonable unit for mandarin speech recognition. On
DidiCallcenter task, Chinese character attention model achieves a CER of 5.68%
and CTC model gets a CER of 7.29%, on the other DidiReading task, CER are 4.89%
and 5.79%, respectively. Moreover, attention model achieves a better
performance than CTC model on both datasets.
Index Set Fourier Series Features for Approximating Multi-dimensional Periodic Kernels
Authors: Anthony Tompkins, Fabio Ramos
Date: 2018-05-14 00:00:00
Tasks: Gaussian Processes

Periodicity is often studied in timeseries modelling with autoregressive
methods but is less popular in the kernel literature, particularly for higher
dimensional problems such as in textures, crystallography, and quantum
mechanics. Large datasets often make modelling periodicity untenable for
otherwise powerful non-parametric methods like Gaussian Processes (GPs) which
typically incur an $\mathcal{O}(N^3)$ computational burden and, consequently,
are unable to scale to larger datasets. To this end we introduce a method
termed \emph{Index Set Fourier Series Features} to tractably exploit
multivariate Fourier series and efficiently decompose periodic kernels on
higher-dimensional data into a series of basis functions. We show that our
approximation produces significantly less predictive error than alternative
approaches such as those based on random Fourier features and achieves better
generalisation on regression problems with periodic data.
Unifying and Merging Well-trained Deep Neural Networks for Inference Stage
Authors: Yi-Min Chou, Yi-Ming Chan, Jia-Hong Lee, Chih-Yi Chiu, Chu-Song Chen
Date: 2018-05-14 00:00:00

We propose a novel method to merge convolutional neural-nets for the
inference stage. Given two well-trained networks that may have different
architectures that handle different tasks, our method aligns the layers of the
original networks and merges them into a unified model by sharing the
representative codes of weights. The shared weights are further re-trained to
fine-tune the performance of the merged model. The proposed method effectively
produces a compact model that may run original tasks simultaneously on
resource-limited devices. As it preserves the general architectures and
leverages the co-used weights of well-trained networks, a substantial training
overhead can be reduced to shorten the system development time. Experimental
results demonstrate a satisfactory performance and validate the effectiveness
of the method.
ContextVP: Fully Context-Aware Video Prediction
Authors: Wonmin Byeon, Qin Wang, Rupesh Kumar Srivastava, Petros Koumoutsakos
Date: 2017-10-23 00:00:00
Proceeding: ECCV 2018 9
Tasks: Prediction, Video Prediction

Video prediction models based on convolutional networks, recurrent networks,
and their combinations often result in blurry predictions. We identify an
important contributing factor for imprecise predictions that has not been
studied adequately in the literature: blind spots, i.e., lack of access to all
relevant past information for accurately predicting the future. To address this
issue, we introduce a fully context-aware architecture that captures the entire
available past context for each pixel using Parallel Multi-Dimensional LSTM
units and aggregates it using blending units. Our model outperforms a strong
baseline network of 20 recurrent convolutional layers and yields
state-of-the-art performance for next step prediction on three challenging
real-world video datasets: Human 3.6M, Caltech Pedestrian, and UCF-101.
Moreover, it does so with fewer parameters than several recently proposed
models, and does not rely on deep convolutional networks, multi-scale
architectures, separation of background and foreground modeling, motion flow
learning, or adversarial training. These results highlight that full awareness
of past context is of crucial importance for video prediction.
Sentiment Adaptive End-to-End Dialog Systems
Authors: Weiyan Shi, Zhou Yu
Date: 2018-04-28 00:00:00
Proceeding: ACL 2018 7
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

End-to-end learning framework is useful for building dialog systems for its simplicity in training and efficiency in model updating. However, current end-to-end approaches only consider user semantic inputs in learning and under-utilize other user information. Therefore, we propose to include user sentiment obtained through multimodal information (acoustic, dialogic and textual), in the end-to-end learning framework to make systems more user-adaptive and effective. We incorporated user sentiment information in both supervised and reinforcement learning settings. In both settings, adding sentiment information reduced the dialog length and improved the task success rate on a bus information search task. This work is the first attempt to incorporate multimodal user information in the adaptive end-to-end dialog system training framework and attained state-of-the-art performance.
Learning Temporal Strategic Relationships using Generative Adversarial Imitation Learning
Authors: Tharindu Fernando, Simon Denman, Sridha Sridharan, Clinton Fookes
Date: 2018-05-13 00:00:00
Tasks: Decision Making, Imitation Learning

This paper presents a novel framework for automatic learning of complex
strategies in human decision making. The task that we are interested in is to
better facilitate long term planning for complex, multi-step events. We observe
temporal relationships at the subtask level of expert demonstrations, and
determine the different strategies employed in order to successfully complete a
task. To capture the relationship between the subtasks and the overall goal, we
utilise two external memory modules, one for capturing dependencies within a
single expert demonstration, such as the sequential relationship among
different sub tasks, and a global memory module for modelling task level
characteristics such as best practice employed by different humans based on
their domain expertise. Furthermore, we demonstrate how the hidden state
representation of the memory can be used as a reward signal to smooth the state
transitions, eradicating subtle changes. We evaluate the effectiveness of the
proposed model for an autonomous highway driving application, where we
demonstrate its capability to learn different expert policies and outperform
state-of-the-art methods. The scope in industrial applications extends to any
robotics and automation application which requires learning from complex
demonstrations containing series of subtasks.
Automated design of collective variables using supervised machine learning
Authors: Mohammad M. Sultan, Vijay S. Pande
Date: 2018-02-28 00:00:00
Tasks: BIG-bench Machine Learning

Selection of appropriate collective variables for enhancing sampling of
molecular simulations remains an unsolved problem in computational biophysics.
In particular, picking initial collective variables (CVs) is particularly
challenging in higher dimensions. Which atomic coordinates or transforms there
of from a list of thousands should one pick for enhanced sampling runs? How
does a modeler even begin to pick starting coordinates for investigation? This
remains true even in the case of simple two state systems and only increases in
difficulty for multi-state systems. In this work, we solve the initial CV
problem using a data-driven approach inspired by the filed of supervised
machine learning. In particular, we show how the decision functions in
supervised machine learning (SML) algorithms can be used as initial CVs
(SML_cv) for accelerated sampling. Using solvated alanine dipeptide and
Chignolin mini-protein as our test cases, we illustrate how the distance to the
Support Vector Machines' decision hyperplane, the output probability estimates
from Logistic Regression, the outputs from deep neural network classifiers, and
other classifiers may be used to reversibly sample slow structural transitions.
We discuss the utility of other SML algorithms that might be useful for
identifying CVs for accelerating molecular simulations.
SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
Authors: Yuhang Song, Chao Yang, Yeji Shen, Peng Wang, Qin Huang, C. -C. Jay Kuo
Date: 2018-05-09 00:00:00
Tasks: Image Generation, Image Inpainting, Interactive Segmentation, Segmentation, Semantic Segmentation

In this paper, we focus on image inpainting task, aiming at recovering the
missing area of an incomplete image given the context information. Recent
development in deep generative models enables an efficient end-to-end framework
for image synthesis and inpainting tasks, but existing methods based on
generative models don't exploit the segmentation information to constrain the
object shapes, which usually lead to blurry results on the boundary. To tackle
this problem, we propose to introduce the semantic segmentation information,
which disentangles the inter-class difference and intra-class variation for
image inpainting. This leads to much clearer recovered boundary between
semantically different regions and better texture within semantically
consistent segments. Our model factorizes the image inpainting process into
segmentation prediction (SP-Net) and segmentation guidance (SG-Net) as two
steps, which predict the segmentation labels in the missing area first, and
then generate segmentation guided inpainting results. Experiments on multiple
public datasets show that our approach outperforms existing methods in
optimizing the image inpainting quality, and the interactive segmentation
guidance provides possibilities for multi-modal predictions of image
inpainting.
Multi-Agent Path Finding with Deadlines: Preliminary Results
Authors: Hang Ma, Glenn Wagner, Ariel Felner, Jiaoyang Li, T. K. Satish Kumar, Sven Koenig
Date: 2018-05-13 00:00:00
Tasks: Multi-Agent Path Finding

We formalize the problem of multi-agent path finding with deadlines
(MAPF-DL). The objective is to maximize the number of agents that can reach
their given goal vertices from their given start vertices within a given
deadline, without colliding with each other. We first show that the MAPF-DL
problem is NP-hard to solve optimally. We then present an optimal MAPF-DL
algorithm based on a reduction of the MAPF-DL problem to a flow problem and a
subsequent compact integer linear programming formulation of the resulting
reduced abstracted multi-commodity flow network.
Accelerating Message Passing for MAP with Benders Decomposition
Authors: Julian Yarkony, Shaofei Wang
Date: 2018-05-13 00:00:00

We introduce a novel mechanism to tighten the local polytope relaxation for
MAP inference in Markov random fields with low state space variables. We
consider a surjection of the variables to a set of hyper-variables and apply
the local polytope relaxation over these hyper-variables. The state space of
each individual hyper-variable is constructed to be enumerable while the vector
product of pairs is not easily enumerable making message passing inference
intractable.
  To circumvent the difficulty of enumerating the vector product of state
spaces of hyper-variables we introduce a novel Benders decomposition approach.
This produces an upper envelope describing the message constructed from affine
functions of the individual variables that compose the hyper-variable receiving
the message. The envelope is tight at the minimizers which are shared by the
true message. Benders rows are constructed to be Pareto optimal and are
generated using an efficient procedure targeted for binary problems.
Compressed Sensing of Scanning Transmission Electron Microscopy (STEM) on Non-Rectangular Scans
Authors: Xin Li, Ondrej Dyck, Sergei V. Kalinin, Stephen Jesse
Date: 2018-05-13 00:00:00
Tasks: compressed sensing

Scanning Transmission Electron Microscopy (STEM) has become the main stay for
materials characterization on atomic level, with applications ranging from
visualization of localized and extended defects to mapping order parameter
fields. In the last several years, attention was attracted by potential of STEM
to explore beam induced chemical processes and especially manipulating atomic
motion, enabling atom-by-atom fabrication. These applications, as well as
traditional imaging of beam sensitive materials, necessitate increasing dynamic
range of STEM between imaging and manipulation modes, and increasing absolute
scanning/imaging speeds, that can be achieved by combining sparse sensing
methods with non-rectangular scanning trajectories. Here we developed a general
method for real-time reconstruction of sparsely sampled images from high-speed,
non-invasive and diverse scanning pathways. This approach is demonstrated on
both the synthetic data where ground truth is known and the experimental STEM
data. This work lays the foundation for future tasks such as optimal design of
dose efficient scanning strategies and real-time adaptive inference and control
of e-beam induced atomic fabrication.
Low-pass Recurrent Neural Networks - A memory architecture for longer-term correlation discovery
Authors: Thomas Stepleton, Razvan Pascanu, Will Dabney, Siddhant M. Jayakumar, Hubert Soyer, Remi Munos
Date: 2018-05-13 00:00:00
Tasks: Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) agents performing complex tasks must be able to
remember observations and actions across sizable time intervals. This is
especially true during the initial learning stages, when exploratory behaviour
can increase the delay between specific actions and their effects. Many new or
popular approaches for learning these distant correlations employ
backpropagation through time (BPTT), but this technique requires storing
observation traces long enough to span the interval between cause and effect.
Besides memory demands, learning dynamics like vanishing gradients and slow
convergence due to infrequent weight updates can reduce BPTT's practicality;
meanwhile, although online recurrent network learning is a developing topic,
most approaches are not efficient enough to use as replacements. We propose a
simple, effective memory strategy that can extend the window over which BPTT
can learn without requiring longer traces. We explore this approach empirically
on a few tasks and discuss its implications.
Learning Rich Features for Image Manipulation Detection
Authors: Peng Zhou, Xintong Han, Vlad I. Morariu, Larry S. Davis
Date: 2018-05-13 00:00:00
Proceeding: CVPR 2018 6
Tasks: Image Manipulation, Image Manipulation Detection, object-detection, Object Detection, Steganalysis

Image manipulation detection is different from traditional semantic object
detection because it pays more attention to tampering artifacts than to image
content, which suggests that richer features need to be learned. We propose a
two-stream Faster R-CNN network and train it endto- end to detect the tampered
regions given a manipulated image. One of the two streams is an RGB stream
whose purpose is to extract features from the RGB image input to find tampering
artifacts like strong contrast difference, unnatural tampered boundaries, and
so on. The other is a noise stream that leverages the noise features extracted
from a steganalysis rich model filter layer to discover the noise inconsistency
between authentic and tampered regions. We then fuse features from the two
streams through a bilinear pooling layer to further incorporate spatial
co-occurrence of these two modalities. Experiments on four standard image
manipulation datasets demonstrate that our two-stream framework outperforms
each individual stream, and also achieves state-of-the-art performance compared
to alternative methods with robustness to resizing and compression.
DeLS-3D: Deep Localization and Segmentation with a 3D Semantic Map
Authors: Peng Wang, Ruigang Yang, Binbin Cao, Wei Xu, Yuanqing Lin
Date: 2018-05-13 00:00:00
Proceeding: CVPR 2018 6
Tasks: Autonomous Driving, Camera Pose Estimation, Pose Estimation, Scene Parsing, Sensor Fusion

For applications such as autonomous driving, self-localization/camera pose
estimation and scene parsing are crucial technologies. In this paper, we
propose a unified framework to tackle these two problems simultaneously. The
uniqueness of our design is a sensor fusion scheme which integrates camera
videos, motion sensors (GPS/IMU), and a 3D semantic map in order to achieve
robustness and efficiency of the system. Specifically, we first have an initial
coarse camera pose obtained from consumer-grade GPS/IMU, based on which a label
map can be rendered from the 3D semantic map. Then, the rendered label map and
the RGB image are jointly fed into a pose CNN, yielding a corrected camera
pose. In addition, to incorporate temporal information, a multi-layer recurrent
neural network (RNN) is further deployed improve the pose accuracy. Finally,
based on the pose from RNN, we render a new label map, which is fed together
with the RGB image into a segment CNN which produces per-pixel semantic label.
In order to validate our approach, we build a dataset with registered 3D point
clouds and video camera images. Both the point clouds and the images are
semantically-labeled. Each video frame has ground truth pose from highly
accurate motion sensors. We show that practically, pose estimation solely
relying on images like PoseNet may fail due to street view confusion, and it is
important to fuse multiple sensors. Finally, various ablation studies are
performed, which demonstrate the effectiveness of the proposed system. In
particular, we show that scene parsing and pose estimation are mutually
beneficial to achieve a more robust and accurate system.
The Global Optimization Geometry of Shallow Linear Neural Networks
Authors: Zhihui Zhu, Daniel Soudry, Yonina C. Eldar, Michael B. Wakin
Date: 2018-05-13 00:00:00
Tasks: global-optimization

We examine the squared error loss landscape of shallow linear neural
networks. We show---with significantly milder assumptions than previous
works---that the corresponding optimization problems have benign geometric
properties: there are no spurious local minima and the Hessian at every saddle
point has at least one negative eigenvalue. This means that at every saddle
point there is a directional negative curvature which algorithms can utilize to
further decrease the objective value. These geometric properties imply that
many local search algorithms (such as the gradient descent which is widely
utilized for training neural networks) can provably solve the training problem
with global convergence.
Dyna: A Method of Momentum for Stochastic Optimization
Authors: Zhidong Han
Date: 2018-05-13 00:00:00
Tasks: Computational Efficiency, Stochastic Optimization

An algorithm is presented for momentum gradient descent optimization based on
the first-order differential equation of the Newtonian dynamics. The fictitious
mass is introduced to the dynamics of momentum for regularizing the adaptive
stepsize of each individual parameter. The dynamic relaxation is adapted for
stochastic optimization of nonlinear objective functions through an explicit
time integration with varying damping ratio. The adaptive stepsize is optimized
for each individual neural network layer based on the number of inputs. The
adaptive stepsize for every parameter over the entire neural network is
uniformly optimized with one upper bound, independent of sparsity, for better
overall convergence rate. The numerical implementation of the algorithm is
similar to the Adam Optimizer, possessing computational efficiency, similar
memory requirements, etc. There are three hyper-parameters in the algorithm
with clear physical interpretation. Preliminary trials show promise in
performance and convergence.
Image Captioning
Authors: Vikram Mullachery, Vishal Motwani
Date: 2018-05-13 00:00:00
Tasks: General Classification, Image Captioning

This paper discusses and demonstrates the outcomes from our experimentation
on Image Captioning. Image captioning is a much more involved task than image
recognition or classification, because of the additional challenge of
recognizing the interdependence between the objects/concepts in the image and
the creation of a succinct sentential narration. Experiments on several labeled
datasets show the accuracy of the model and the fluency of the language it
learns solely from image descriptions. As a toy application, we apply image
captioning to create video captions, and we advance a few hypotheses on the
challenges we encountered.
Lehmer Transform and its Theoretical Properties
Authors: Masoud Ataei, Shengyuan Chen, Xiaogang Wang
Date: 2018-05-13 00:00:00
Tasks: EEG, Electroencephalogram (EEG)

We propose a new class of transforms that we call {\it Lehmer Transform}
which is motivated by the {\it Lehmer mean function}. The proposed {\it Lehmer
transform} decomposes a function of a sample into their constituting
statistical moments. Theoretical properties of the proposed transform are
presented. This transform could be very useful to provide an alternative method
in analyzing non-stationary signals such as brain wave EEG.