pwc-archive/papers-with-abstracts huggingface dataset:
Attention-based Memory Selection Recurrent Network for Language Modeling
Authors: Da-Rong Liu, Shun-Po Chuang, Hung-Yi Lee
Date: 2016-11-26 00:00:00
Tasks: Language Modeling, Language Modelling, Sentence

Recurrent neural networks (RNNs) have achieved great success in language
modeling. However, since the RNNs have fixed size of memory, their memory
cannot store all the information about the words it have seen before in the
sentence, and thus the useful long-term information may be ignored when
predicting the next words. In this paper, we propose Attention-based Memory
Selection Recurrent Network (AMSRN), in which the model can review the
information stored in the memory at each previous time step and select the
relevant information to help generate the outputs. In AMSRN, the attention
mechanism finds the time steps storing the relevant information in the memory,
and memory selection determines which dimensions of the memory are involved in
computing the attention weights and from which the information is extracted.In
the experiments, AMSRN outperformed long short-term memory (LSTM) based
language models on both English and Chinese corpora. Moreover, we investigate
using entropy as a regularizer for attention weights and visualize how the
attention mechanism helps language modeling.
A Deep Neural Network to identify foreshocks in real time
Authors: K. Vikraman
Date: 2016-11-26 00:00:00
Tasks: General Classification

Foreshock events provide valuable insight to predict imminent major
earthquakes. However, it is difficult to identify them in real time. In this
paper, I propose an algorithm based on deep learning to instantaneously
classify a seismic waveform as a foreshock, mainshock or an aftershock event
achieving a high accuracy of 99% in classification. As a result, this is by far
the most reliable method to predict major earthquakes that are preceded by
foreshocks. In addition, I discuss methods to create an earthquake dataset that
is compatible with deep networks.
Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition
Authors: Zeyuan Allen-Zhu, Yuanzhi Li
Date: 2016-07-20 00:00:00
Proceeding: ICML 2017 8

We study $k$-GenEV, the problem of finding the top $k$ generalized
eigenvectors, and $k$-CCA, the problem of finding the top $k$ vectors in
canonical-correlation analysis. We propose algorithms $\mathtt{LazyEV}$ and
$\mathtt{LazyCCA}$ to solve the two problems with running times linearly
dependent on the input size and on $k$.
  Furthermore, our algorithms are DOUBLY-ACCELERATED: our running times depend
only on the square root of the matrix condition number, and on the square root
of the eigengap. This is the first such result for both $k$-GenEV or $k$-CCA.
We also provide the first gap-free results, which provide running times that
depend on $1/\sqrt{\varepsilon}$ rather than the eigengap.
Word Embedding based Correlation Model for Question/Answer Matching
Authors: Yikang Shen, Wenge Rong, Nan Jiang, Baolin Peng, Jie Tang, Zhang Xiong
Date: 2015-11-15 00:00:00
Tasks: Question Answering, Translation

With the development of community based question answering (Q&A) services, a
large scale of Q&A archives have been accumulated and are an important
information and knowledge resource on the web. Question and answer matching has
been attached much importance to for its ability to reuse knowledge stored in
these systems: it can be useful in enhancing user experience with recurrent
questions. In this paper, we try to improve the matching accuracy by overcoming
the lexical gap between question and answer pairs. A Word Embedding based
Correlation (WEC) model is proposed by integrating advantages of both the
translation model and word embedding, given a random pair of words, WEC can
score their co-occurrence probability in Q&A pairs and it can also leverage the
continuity and smoothness of continuous space word representation to deal with
new pairs of words that are rare in the training parallel text. An experimental
study on Yahoo! Answers dataset and Baidu Zhidao dataset shows this new
method's promising potential.
Learning to Draw Samples: With Application to Amortized MLE for Generative Adversarial Learning
Authors: Dilin Wang, Qiang Liu
Date: 2016-11-06 00:00:00
Tasks: Conditional Image Generation

We propose a simple algorithm to train stochastic neural networks to draw
samples from given target distributions for probabilistic inference. Our method
is based on iteratively adjusting the neural network parameters so that the
output changes along a Stein variational gradient that maximumly decreases the
KL divergence with the target distribution. Our method works for any target
distribution specified by their unnormalized density function, and can train
any black-box architectures that are differentiable in terms of the parameters
we want to adapt. As an application of our method, we propose an amortized MLE
algorithm for training deep energy model, where a neural sampler is adaptively
trained to approximate the likelihood function. Our method mimics an
adversarial game between the deep energy model and the neural sampler, and
obtains realistic-looking images competitive with the state-of-the-art results.
Learning in Quantum Control: High-Dimensional Global Optimization for Noisy Quantum Dynamics
Authors: Pantita Palittapongarnpim, Peter Wittek, Ehsan Zahedinejad, Shakib Vedaie, Barry C. Sanders
Date: 2016-07-12 00:00:00
Tasks: global-optimization, quantum gate design, Reinforcement Learning

Quantum control is valuable for various quantum technologies such as
high-fidelity gates for universal quantum computing, adaptive quantum-enhanced
metrology, and ultra-cold atom manipulation. Although supervised machine
learning and reinforcement learning are widely used for optimizing control
parameters in classical systems, quantum control for parameter optimization is
mainly pursued via gradient-based greedy algorithms. Although the quantum
fitness landscape is often compatible with greedy algorithms, sometimes greedy
algorithms yield poor results, especially for large-dimensional quantum
systems. We employ differential evolution algorithms to circumvent the
stagnation problem of non-convex optimization. We improve quantum control
fidelity for noisy system by averaging over the objective function. To reduce
computational cost, we introduce heuristics for early termination of runs and
for adaptive selection of search subspaces. Our implementation is massively
parallel and vectorized to reduce run time even further. We demonstrate our
methods with two examples, namely quantum phase estimation and quantum gate
design, for which we achieve superior fidelity and scalability than obtained
using greedy algorithms.
Student's t Distribution based Estimation of Distribution Algorithms for Derivative-free Global Optimization
Authors: Bin Liu, Shi Cheng, Yuhui Shi
Date: 2016-08-12 00:00:00
Tasks: Evolutionary Algorithms, global-optimization

In this paper, we are concerned with a branch of evolutionary algorithms
termed estimation of distribution (EDA), which has been successfully used to
tackle derivative-free global optimization problems. For existent EDA
algorithms, it is a common practice to use a Gaussian distribution or a mixture
of Gaussian components to represent the statistical property of available
promising solutions found so far. Observing that the Student's t distribution
has heavier and longer tails than the Gaussian, which may be beneficial for
exploring the solution space, we propose a novel EDA algorithm termed ESTDA, in
which the Student's t distribution, rather than Gaussian, is employed. To
address hard multimodal and deceptive problems, we extend ESTDA further by
substituting a single Student's t distribution with a mixture of Student's t
distributions. The resulting algorithm is named as estimation of mixture of
Student's t distribution algorithm (EMSTDA). Both ESTDA and EMSTDA are
evaluated through extensive and in-depth numerical experiments using over a
dozen of benchmark objective functions. Empirical results demonstrate that the
proposed algorithms provide remarkably better performance than their Gaussian
counterparts.
Texture analysis using deterministic partially self-avoiding walk with thresholds
Authors: Lucas Correia Ribas, Wesley Nunes Gonçalves, Odemir Martinez Bruno
Date: 2016-11-25 00:00:00
Tasks: Texture Classification

In this paper, we propose a new texture analysis method using the
deterministic partially self-avoiding walk performed on maps modified with
thresholds. In this method, two pixels of the map are neighbors if the
Euclidean distance between them is less than $\sqrt{2}$ and the weight
(difference between its intensities) is less than a given threshold. The maps
obtained by using different thresholds highlight several properties of the
image that are extracted by the deterministic walk. To compose the feature
vector, deterministic walks are performed with different thresholds and its
statistics are concatenated. Thus, this approach can be considered as a
multi-scale analysis. We validate our method on the Brodatz database, which is
very well known public image database and widely used by texture analysis
methods. Experimental results indicate that the proposed method presents a good
texture discrimination, overcoming traditional texture methods.
Directional Mean Curvature for Textured Image Demixing
Authors: Duy Hoang Thai, David Banks
Date: 2016-11-25 00:00:00
Tasks: Image Deconvolution

Approximation theory plays an important role in image processing, especially
image deconvolution and decomposition. For piecewise smooth images, there are
many methods that have been developed over the past thirty years. The goal of
this study is to devise similar and practical methodology for handling textured
images. This problem is motivated by forensic imaging, since fingerprints,
shoeprints and bullet ballistic evidence are textured images. In particular, it
is known that texture information is almost destroyed by a blur operator, such
as a blurred ballistic image captured from a low-cost microscope. The
contribution of this work is twofold: first, we propose a mathematical model
for textured image deconvolution and decomposition into four meaningful
components, using a high-order partial differential equation approach based on
the directional mean curvature. Second, we uncover a link between functional
analysis and multiscale sampling theory, e.g., harmonic analysis and filter
banks. Both theoretical results and examples with natural images are provided
to illustrate the performance of the proposed model.
Fast deterministic tourist walk for texture analysis
Authors: Lucas Correia Ribas, Odemir Martinez Bruno
Date: 2016-11-25 00:00:00
Tasks: General Classification, Texture Classification

Deterministic tourist walk (DTW) has attracted increasing interest in
computer vision. In the last years, different methods for analysis of dynamic
and static textures were proposed. So far, all works based on the DTW for
texture analysis use all image pixels as initial point of a walk. However, this
requires much runtime. In this paper, we conducted a study to verify the
performance of the DTW method according to the number of initial points to
start a walk. The proposed method assigns a unique code to each image pixel,
then, the pixels whose code is not divisible by a given $k$ value are ignored
as initial points of walks. Feature vectors were extracted and a classification
process was performed for different percentages of initial points. Experimental
results on the Brodatz and Vistex datasets indicate that to use fewer pixels as
initial points significantly improves the runtime compared to use all image
pixels. In addition, the correct classification rate decreases very little.
A Benchmark and Comparison of Active Learning for Logistic Regression
Authors: Yazhou Yang, Marco Loog
Date: 2016-11-25 00:00:00
Tasks: Active Learning, General Classification, regression

Logistic regression is by far the most widely used classifier in real-world
applications. In this paper, we benchmark the state-of-the-art active learning
methods for logistic regression and discuss and illustrate their underlying
characteristics. Experiments are carried out on three synthetic datasets and 44
real-world datasets, providing insight into the behaviors of these active
learning methods with respect to the area of the learning curve (which plots
classification accuracy as a function of the number of queried examples) and
their computational costs. Surprisingly, one of the earliest and simplest
suggested active learning methods, i.e., uncertainty sampling, performs
exceptionally well overall. Another remarkable finding is that random sampling,
which is the rudimentary baseline to improve upon, is not overwhelmed by
individual active learning techniques in many cases.
Multi-Object Reasoning with Constrained Goal Models
Authors: Chi Mai Nguyen, Roberto Sebastiani, Paolo Giorgini, John Mylopoulos
Date: 2016-01-27 00:00:00
Tasks: Object

Goal models have been widely used in Computer Science to represent software
requirements, business objectives, and design qualities. Existing goal
modelling techniques, however, have shown limitations of expressiveness and/or
tractability in coping with complex real-world problems. In this work, we
exploit advances in automated reasoning technologies, notably Satisfiability
and Optimization Modulo Theories (SMT/OMT), and we propose and formalize: (i)
an extended modelling language for goals, namely the Constrained Goal Model
(CGM), which makes explicit the notion of goal refinement and of domain
assumption, allows for expressing preferences between goals and refinements,
and allows for associating numerical attributes to goals and refinements for
defining constraints and optimization goals over multiple objective functions,
refinements and their numerical attributes; (ii) a novel set of automated
reasoning functionalities over CGMs, allowing for automatically generating
suitable refinements of input CGMs, under user-specified assumptions and
constraints, that also maximize preferences and optimize given objective
functions. We have implemented these modelling and reasoning functionalities in
a tool, named CGM-Tool, using the OMT solver OptiMathSAT as automated reasoning
backend. Moreover, we have conducted an experimental evaluation on large CGMs
to support the claim that our proposal scales well for goal models with
thousands of elements.
A Neural Network Model to Classify Liver Cancer Patients Using Data Expansion and Compression
Authors: Ashkan Zeinalzadeh, Tom Wenska, Gordon Okimoto
Date: 2016-11-23 00:00:00

We develop a neural network model to classify liver cancer patients into
high-risk and low-risk groups using genomic data. Our approach provides a novel
technique to classify big data sets using neural network models. We preprocess
the data before training the neural network models. We first expand the data
using wavelet analysis. We then compress the wavelet coefficients by mapping
them onto a new scaled orthonormal coordinate system. Then the data is used to
train a neural network model that enables us to classify cancer patients into
two different classes of high-risk and low-risk patients. We use the
leave-one-out approach to build a neural network model. This neural network
model enables us to classify a patient using genomic data as a high-risk or
low-risk patient without any information about the survival time of the
patient. The results from genomic data analysis are compared with survival time
analysis. It is shown that the expansion and compression of data using wavelet
analysis and singular value decomposition (SVD) is essential to train the
neural network model.
PVR: Patch-to-Volume Reconstruction for Large Area Motion Correction of Fetal MRI
Authors: Amir Alansary, Bernhard Kainz, Martin Rajchl, Maria Murgasova, Mellisa Damodaram, David F. A. Lloyd, Alice Davidson, Steven G. McDonagh, Mary Rutherford, Joseph V. Hajnal, Daniel Rueckert
Date: 2016-11-22 00:00:00
Tasks: GPU, Motion Compensation, SSIM, Super-Resolution

In this paper we present a novel method for the correction of motion
artifacts that are present in fetal Magnetic Resonance Imaging (MRI) scans of
the whole uterus. Contrary to current slice-to-volume registration (SVR)
methods, requiring an inflexible anatomical enclosure of a single investigated
organ, the proposed patch-to-volume reconstruction (PVR) approach is able to
reconstruct a large field of view of non-rigidly deforming structures. It
relaxes rigid motion assumptions by introducing a specific amount of redundant
information that is exploited with parallelized patch-wise optimization,
super-resolution, and automatic outlier rejection. We further describe and
provide an efficient parallel implementation of PVR allowing its execution
within reasonable time on commercially available graphics processing units
(GPU), enabling its use in the clinical practice. We evaluate PVR's
computational overhead compared to standard methods and observe improved
reconstruction accuracy in presence of affine motion artifacts of approximately
30% compared to conventional SVR in synthetic experiments. Furthermore, we have
evaluated our method qualitatively and quantitatively on real fetal MRI data
subject to maternal breathing and sudden fetal movements. We evaluate
peak-signal-to-noise ratio (PSNR), structural similarity index (SSIM), and
cross correlation (CC) with respect to the originally acquired data and provide
a method for visual inspection of reconstruction uncertainty. With these
experiments we demonstrate successful application of PVR motion compensation to
the whole uterus, the human fetus, and the human placenta.
QBF Solving by Counterexample-guided Expansion
Authors: Roderick Bloem, Nicolas Braud-Santoni, Vedad Hadzic
Date: 2016-11-04 00:00:00

We introduce a novel generalization of Counterexample-Guided Inductive
Synthesis (CEGIS) and instantiate it to yield a novel, competitive algorithm
for solving Quantified Boolean Formulas (QBF). Current QBF solvers based on
counterexample-guided expansion use a recursive approach which scales poorly
with the number of quantifier alternations. Our generalization of CEGIS removes
the need for this recursive approach, and we instantiate it to yield a simple
and efficient algorithm for QBF solving. Lastly, this research is supported by
a competitive, though straightforward, implementation of the algorithm, making
it possible to study the practical impact of our algorithm design decisions,
along with various optimizations.
End-to-End Neural Sentence Ordering Using Pointer Network
Authors: Jingjing Gong, Xinchi Chen, Xipeng Qiu, Xuanjing Huang
Date: 2016-11-15 00:00:00
Tasks: Sentence, Sentence Ordering

Sentence ordering is one of important tasks in NLP. Previous works mainly
focused on improving its performance by using pair-wise strategy. However, it
is nontrivial for pair-wise models to incorporate the contextual sentence
information. In addition, error prorogation could be introduced by using the
pipeline strategy in pair-wise models. In this paper, we propose an end-to-end
neural approach to address the sentence ordering problem, which uses the
pointer network (Ptr-Net) to alleviate the error propagation problem and
utilize the whole contextual information. Experimental results show the
effectiveness of the proposed model. Source codes and dataset of this paper are
available.
Dictionary Integration using 3D Morphable Face Models for Pose-invariant Collaborative-representation-based Classification
Authors: Xiaoning Song, Zhen-Hua Feng, Guosheng Hu, Josef Kittler, William Christmas, Xiao-Jun Wu
Date: 2016-11-01 00:00:00
Tasks: Classification, General Classification

The paper presents a dictionary integration algorithm using 3D morphable face
models (3DMM) for pose-invariant collaborative-representation-based face
classification. To this end, we first fit a 3DMM to the 2D face images of a
dictionary to reconstruct the 3D shape and texture of each image. The 3D faces
are used to render a number of virtual 2D face images with arbitrary pose
variations to augment the training data, by merging the original and rendered
virtual samples to create an extended dictionary. Second, to reduce the
information redundancy of the extended dictionary and improve the sparsity of
reconstruction coefficient vectors using collaborative-representation-based
classification (CRC), we exploit an on-line elimination scheme to optimise the
extended dictionary by identifying the most representative training samples for
a given query. The final goal is to perform pose-invariant face classification
using the proposed dictionary integration method and the on-line pruning
strategy under the CRC framework. Experimental results obtained for a set of
well-known face datasets demonstrate the merits of the proposed method,
especially its robustness to pose variations.
Label Efficient Learning by Exploiting Multi-class Output Codes
Authors: Maria Florina Balcan, Travis Dick, Yishay Mansour
Date: 2015-11-10 00:00:00

We present a new perspective on the popular multi-class algorithmic
techniques of one-vs-all and error correcting output codes. Rather than
studying the behavior of these techniques for supervised learning, we establish
a connection between the success of these methods and the existence of
label-efficient learning procedures. We show that in both the realizable and
agnostic cases, if output codes are successful at learning from labeled data,
they implicitly assume structure on how the classes are related. By making that
structure explicit, we design learning algorithms to recover the classes with
low label complexity. We provide results for the commonly studied cases of
one-vs-all learning and when the codewords of the classes are well separated.
We additionally consider the more challenging case where the codewords are not
well separated, but satisfy a boundary features condition that captures the
natural intuition that every bit of the codewords should be significant.
On the Exponentially Weighted Aggregate with the Laplace Prior
Authors: Arnak S. Dalalyan, Edwin Grappin, Quentin Paris
Date: 2016-11-25 00:00:00
Tasks: regression

In this paper, we study the statistical behaviour of the Exponentially
Weighted Aggregate (EWA) in the problem of high-dimensional regression with
fixed design. Under the assumption that the underlying regression vector is
sparse, it is reasonable to use the Laplace distribution as a prior. The
resulting estimator and, specifically, a particular instance of it referred to
as the Bayesian lasso, was already used in the statistical literature because
of its computational convenience, even though no thorough mathematical analysis
of its statistical properties was carried out. The present work fills this gap
by establishing sharp oracle inequalities for the EWA with the Laplace prior.
These inequalities show that if the temperature parameter is small, the EWA
with the Laplace prior satisfies the same type of oracle inequality as the
lasso estimator does, as long as the quality of estimation is measured by the
prediction loss. Extensions of the proposed methodology to the problem of
prediction with low-rank matrices are considered.
Multimodal Latent Variable Analysis
Authors: Vardan Papyan, Ronen Talmon
Date: 2016-11-25 00:00:00

Consider a set of multiple, multimodal sensors capturing a complex system or
a physical phenomenon of interest. Our primary goal is to distinguish the
underlying sources of variability manifested in the measured data. The first
step in our analysis is to find the common source of variability present in all
sensor measurements. We base our work on a recent paper, which tackles this
problem with alternating diffusion (AD). In this work, we suggest to further
the analysis by extracting the sensor-specific variables in addition to the
common source. We propose an algorithm, which we analyze theoretically, and
then demonstrate on three different applications: a synthetic example, a toy
problem, and the task of fetal ECG extraction.