pwc-archive/papers-with-abstracts huggingface dataset:
Can neural machine translation do simultaneous translation?
Authors: Kyunghyun Cho, Masha Esipova
Date: 2016-06-07 00:00:00
Tasks: Machine Translation, Sentence, Translation

We investigate the potential of attention-based neural machine translation in
simultaneous translation. We introduce a novel decoding algorithm, called
simultaneous greedy decoding, that allows an existing neural machine
translation model to begin translating before a full source sentence is
received. This approach is unique from previous works on simultaneous
translation in that segmentation and translation are done jointly to maximize
the translation quality and that translating each segment is strongly
conditioned on all the previous segments. This paper presents a first step
toward building a full simultaneous translation system based on neural machine
translation.
Memory-enhanced Decoder for Neural Machine Translation
Authors: Mingxuan Wang, Zhengdong Lu, Hang Li, Qun Liu
Date: 2016-06-07 00:00:00
Proceeding: EMNLP 2016 11
Tasks: Decoder, Machine Translation, NMT, Sentence, Translation

We propose to enhance the RNN decoder in a neural machine translator (NMT)
with external memory, as a natural but powerful extension to the state in the
decoding RNN. This memory-enhanced RNN decoder is called \textsc{MemDec}. At
each time during decoding, \textsc{MemDec} will read from this memory and write
to this memory once, both with content-based addressing. Unlike the unbounded
memory in previous work\cite{RNNsearch} to store the representation of source
sentence, the memory in \textsc{MemDec} is a matrix with pre-determined size
designed to better capture the information important for the decoding process
at each time step. Our empirical study on Chinese-English translation shows
that it can improve by $4.8$ BLEU upon Groundhog and $5.3$ BLEU upon on Moses,
yielding the best performance achieved with the same training set.
Matrix Factorization using Window Sampling and Negative Sampling for Improved Word Representations
Authors: Alexandre Salle, Marco Idiart, Aline Villavicencio
Date: 2016-06-02 00:00:00
Proceeding: ACL 2016 8
Tasks: Word Similarity

In this paper, we propose LexVec, a new method for generating distributed
word representations that uses low-rank, weighted factorization of the Positive
Point-wise Mutual Information matrix via stochastic gradient descent, employing
a weighting scheme that assigns heavier penalties for errors on frequent
co-occurrences while still accounting for negative co-occurrence. Evaluation on
word similarity and analogy tasks shows that LexVec matches and often
outperforms state-of-the-art methods on many of these tasks.
Neural Network Models for Implicit Discourse Relation Classification in English and Chinese without Surface Features
Authors: Attapol T. Rutherford, Vera Demberg, Nianwen Xue
Date: 2016-06-07 00:00:00
Tasks: Discourse Parsing, General Classification, Implicit Discourse Relation Classification, Relation, Relation Classification

Inferring implicit discourse relations in natural language text is the most
difficult subtask in discourse parsing. Surface features achieve good
performance, but they are not readily applicable to other languages without
semantic lexicons. Previous neural models require parses, surface features, or
a small label set to work well. Here, we propose neural network models that are
based on feedforward and long-short term memory architecture without any
surface features. To our surprise, our best configured feedforward architecture
outperforms LSTM-based model in most cases despite thorough tuning. Under
various fine-grained label sets and a cross-linguistic setting, our feedforward
models perform consistently better or at least just as well as systems that
require hand-crafted surface features. Our models present the first neural
Chinese discourse parser in the style of Chinese Discourse Treebank, showing
that our results hold cross-linguistically.
Curie: A method for protecting SVM Classifier from Poisoning Attack
Authors: Ricky Laishram, Vir Virander Phoha
Date: 2016-06-05 00:00:00
Tasks: BIG-bench Machine Learning, Speaker Identification, valid

Machine learning is used in a number of security related applications such as
biometric user authentication, speaker identification etc. A type of causative
integrity attack against machine learning called Poisoning attack works by
injecting specially crafted data points in the training data so as to increase
the false positive rate of the classifier. In the context of the biometric
authentication, this means that more intruders will be classified as valid
user, and in case of speaker identification system, user A will be classified
user B. In this paper, we examine poisoning attack against SVM and introduce -
Curie - a method to protect the SVM classifier from the poisoning attack. The
basic idea of our method is to identify the poisoned data points injected by
the adversary and filter them out. Our method is light weight and can be easily
integrated into existing systems. Experimental results show that it works very
well in filtering out the poisoned data.
Deep neural networks are robust to weight binarization and other non-linear distortions
Authors: Paul Merolla, Rathinakumar Appuswamy, John Arthur, Steve K. Esser, Dharmendra Modha
Date: 2016-06-07 00:00:00
Tasks: Binarization, Data Augmentation, Quantization

Recent results show that deep neural networks achieve excellent performance
even when, during training, weights are quantized and projected to a binary
representation. Here, we show that this is just the tip of the iceberg: these
same networks, during testing, also exhibit a remarkable robustness to
distortions beyond quantization, including additive and multiplicative noise,
and a class of non-linear projections where binarization is just a special
case. To quantify this robustness, we show that one such network achieves 11%
test error on CIFAR-10 even with 0.68 effective bits per weight. Furthermore,
we find that a common training heuristic--namely, projecting quantized weights
during backpropagation--can be altered (or even removed) and networks still
achieve a base level of robustness during testing. Specifically, training with
weight projections other than quantization also works, as does simply clipping
the weights, both of which have never been reported before. We confirm our
results for CIFAR-10 and ImageNet datasets. Finally, drawing from these ideas,
we propose a stochastic projection rule that leads to a new state of the art
network with 7.64% test error on CIFAR-10 using no data augmentation.
Assisted Energy Management in Smart Microgrids
Authors: Andrea Monacchi, Wilfried Elmenreich
Date: 2016-06-06 00:00:00
Tasks: energy management, Management

Demand response provides utilities with a mechanism to share with end users
the stochasticity resulting from the use of renewable sources. Pricing is
accordingly used to reflect energy availability, to allocate such a limited
resource to those loads that value it most. However, the strictly competitive
mechanism can result in service interruption in presence of competing demand.
To solve this issue we investigate on the use of forward contracts, i.e.,
service level agreements priced to reflect the expectation of future supply and
demand curves. Given the limited resources of microgrids, service interruption
is an opposite objective to the one of service availability. We firstly design
policy-based brokers and identify then a learning broker based on artificial
neural networks. We show the latter being progressively minimizing the
reimbursement costs and maximizing the overall profit.
Language to Logical Form with Neural Attention
Authors: Li Dong, Mirella Lapata
Date: 2016-01-06 00:00:00
Proceeding: ACL 2016 8
Tasks: Decoder, Form, Semantic Parsing

Semantic parsing aims at mapping natural language to machine interpretable
meaning representations. Traditional approaches rely on high-quality lexicons,
manually-built templates, and linguistic features which are either domain- or
representation-specific. In this paper we present a general method based on an
attention-enhanced encoder-decoder model. We encode input utterances into
vector representations, and generate their logical forms by conditioning the
output sequences or trees on the encoding vectors. Experimental results on four
datasets show that our approach performs competitively without using
hand-engineered features and is easy to adapt across domains and meaning
representations.
Consistency and Trust in Peer Data Exchange Systems
Authors: Leopoldo Bertossi, Loreto Bravo
Date: 2016-06-06 00:00:00

We propose and investigate a semantics for "peer data exchange systems" where
different peers are related by data exchange constraints and trust
relationships. These two elements plus the data at the peers' sites and their
local integrity constraints are made compatible via a semantics that
characterizes sets of "solution instances" for the peers. They are the intended
-possibly virtual- instances for a peer that are obtained through a data repair
semantics that we introduce and investigate. The semantically correct answers
from a peer to a query, the so-called "peer consistent answers", are defined as
those answers that are invariant under all its different solution instances. We
show that solution instances can be specified as the models of logic programs
with a stable model semantics. The repair semantics is based on null values as
used in SQL databases, and is also of independent interest for repairs of
single databases with respect to integrity constraints.
Learning to Optimize
Authors: Ke Li, Jitendra Malik
Date: 2016-06-06 00:00:00
Proceeding: 2016 2016 6
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Algorithm design is a laborious process and often requires many iterations of
ideation and validation. In this paper, we explore automating algorithm design
and present a method to learn an optimization algorithm, which we believe to be
the first method that can automatically discover a better algorithm. We
approach this problem from a reinforcement learning perspective and represent
any particular optimization algorithm as a policy. We learn an optimization
algorithm using guided policy search and demonstrate that the resulting
algorithm outperforms existing hand-engineered algorithms in terms of
convergence speed and/or the final objective value.
Bayesian Poisson Tucker Decomposition for Learning the Structure of International Relations
Authors: Aaron Schein, Mingyuan Zhou, David M. Blei, Hanna Wallach
Date: 2016-06-06 00:00:00

We introduce Bayesian Poisson Tucker decomposition (BPTD) for modeling
country--country interaction event data. These data consist of interaction
events of the form "country $i$ took action $a$ toward country $j$ at time
$t$." BPTD discovers overlapping country--community memberships, including the
number of latent communities. In addition, it discovers directed
community--community interaction networks that are specific to "topics" of
action types and temporal "regimes." We show that BPTD yields an efficient MCMC
inference algorithm and achieves better predictive performance than related
models. We also demonstrate that it discovers interpretable latent structure
that agrees with our knowledge of international relations.
Homology Computation of Large Point Clouds using Quantum Annealing
Authors: Raouf Dridi, Hedayat Alghassi
Date: 2015-12-23 00:00:00
Tasks: Topological Data Analysis

Homology is a tool in topological data analysis which measures the shape of
the data. In many cases, these measurements translate into new insights which
are not available by other means. To compute homology, we rely on mathematical
constructions which scale exponentially with the size of the data. Therefore,
for large point clouds, the computation is infeasible using classical
computers. In this paper, we present a quantum annealing pipeline for
computation of homology of large point clouds. The pipeline takes as input a
graph approximating the given point cloud. It uses quantum annealing to compute
a clique covering of the graph and then uses this cover to construct a
Mayer-Vietoris complex. The pipeline terminates by performing a simplified
homology computation of the Mayer-Vietoris complex. We have introduced three
different clique coverings and their quantum annealing formulation. Our
pipeline scales polynomially in the size of the data, once the covering step is
solved. To prove correctness of our algorithm, we have also included tests
using D-Wave 2X quantum processor.
Learning Non-Parametric Basis Independent Models from Point Queries via Low-Rank Methods
Authors: Hemant Tyagi, Volkan Cevher
Date: 2013-10-07 00:00:00

We consider the problem of learning multi-ridge functions of the form f(x) =
g(Ax) from point evaluations of f. We assume that the function f is defined on
an l_2-ball in R^d, g is twice continuously differentiable almost everywhere,
and A \in R^{k \times d} is a rank k matrix, where k << d. We propose a
randomized, polynomial-complexity sampling scheme for estimating such
functions. Our theoretical developments leverage recent techniques from low
rank matrix recovery, which enables us to derive a polynomial time estimator of
the function f along with uniform approximation guarantees. We prove that our
scheme can also be applied for learning functions of the form: f(x) =
\sum_{i=1}^{k} g_i(a_i^T x), provided f satisfies certain smoothness conditions
in a neighborhood around the origin. We also characterize the noise robustness
of the scheme. Finally, we present numerical examples to illustrate the
theoretical bounds in action.
Online Unmixing of Multitemporal Hyperspectral Images accounting for Spectral Variability
Authors: Pierre-Antoine Thouvenin, Nicolas Dobigeon, Jean-Yves Tourneret
Date: 2015-10-20 00:00:00
Tasks: Hyperspectral Unmixing

Hyperspectral unmixing is aimed at identifying the reference spectral
signatures composing an hyperspectral image and their relative abundance
fractions in each pixel. In practice, the identified signatures may vary
spectrally from an image to another due to varying acquisition conditions, thus
inducing possibly significant estimation errors. Against this background,
hyperspectral unmixing of several images acquired over the same area is of
considerable interest. Indeed, such an analysis enables the endmembers of the
scene to be tracked and the corresponding endmember variability to be
characterized. Sequential endmember estimation from a set of hyperspectral
images is expected to provide improved performance when compared to methods
analyzing the images independently. However, the significant size of
hyperspectral data precludes the use of batch procedures to jointly estimate
the mixture parameters of a sequence of hyperspectral images. Provided that
each elementary component is present in at least one image of the sequence, we
propose to perform an online hyperspectral unmixing accounting for temporal
endmember variability. The online hyperspectral unmixing is formulated as a
two-stage stochastic program, which can be solved using a stochastic
approximation. The performance of the proposed method is evaluated on synthetic
and real data. A comparison with independent unmixing algorithms finally
illustrates the interest of the proposed strategy.
Neural Machine Translation with External Phrase Memory
Authors: Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li, Philip L. H. Yu
Date: 2016-06-06 00:00:00
Tasks: Decoder, Machine Translation, Sentence, Translation

In this paper, we propose phraseNet, a neural machine translator with a
phrase memory which stores phrase pairs in symbolic form, mined from corpus or
specified by human experts. For any given source sentence, phraseNet scans the
phrase memory to determine the candidate phrase pairs and integrates tagging
information in the representation of source sentence accordingly. The decoder
utilizes a mixture of word-generating component and phrase-generating
component, with a specifically designed strategy to generate a sequence of
multiple words all at once. The phraseNet not only approaches one step towards
incorporating external knowledge into neural machine translation, but also
makes an effort to extend the word-by-word generation mechanism of recurrent
neural network. Our empirical study on Chinese-to-English translation shows
that, with carefully-chosen phrase table in memory, phraseNet yields 3.45 BLEU
improvement over the generic neural machine translator.
Dynamic Filter Networks
Authors: Bert De Brabandere, Xu Jia, Tinne Tuytelaars, Luc van Gool
Date: 2016-05-31 00:00:00
Proceeding: NeurIPS 2016 12
Tasks: Depth Estimation, Optical Flow Estimation, Video Prediction

In a traditional convolutional layer, the learned filters stay fixed after
training. In contrast, we introduce a new framework, the Dynamic Filter
Network, where filters are generated dynamically conditioned on an input. We
show that this architecture is a powerful one, with increased flexibility
thanks to its adaptive nature, yet without an excessive increase in the number
of model parameters. A wide variety of filtering operations can be learned this
way, including local spatial transformations, but also others like selective
(de)blurring or adaptive feature extraction. Moreover, multiple such layers can
be combined, e.g. in a recurrent architecture. We demonstrate the effectiveness
of the dynamic filter network on the tasks of video and stereo prediction, and
reach state-of-the-art performance on the moving MNIST dataset with a much
smaller model. By visualizing the learned filters, we illustrate that the
network has picked up flow information by only looking at unlabelled training
data. This suggests that the network can be used to pretrain networks for
various supervised tasks in an unsupervised way, like optical flow and depth
estimation.
Discovering Phase Transitions with Unsupervised Learning
Authors: Lei Wang
Date: 2016-06-01 00:00:00
Tasks: Clustering

Unsupervised learning is a discipline of machine learning which aims at
discovering patterns in big data sets or classifying the data into several
categories without being trained explicitly. We show that unsupervised learning
techniques can be readily used to identify phases and phases transitions of
many body systems. Starting with raw spin configurations of a prototypical
Ising model, we use principal component analysis to extract relevant low
dimensional representations the original data and use clustering analysis to
identify distinct phases in the feature space. This approach successfully finds
out physical concepts such as order parameter and structure factor to be
indicators of the phase transition. We discuss future prospects of discovering
more complex phases and phase transitions using unsupervised learning
techniques.
Recursive n-gram hashing is pairwise independent, at best
Authors: Daniel Lemire, Owen Kaser
Date: 2007-05-31 00:00:00

Many applications use sequences of n consecutive symbols (n-grams). Hashing
these n-grams can be a performance bottleneck. For more speed, recursive hash
families compute hash values by updating previous values. We prove that
recursive hash families cannot be more than pairwise independent. While hashing
by irreducible polynomials is pairwise independent, our implementations either
run in time O(n) or use an exponential amount of memory. As a more scalable
alternative, we make hashing by cyclic polynomials pairwise independent by
ignoring n-1 bits. Experimentally, we show that hashing by cyclic polynomials
is is twice as fast as hashing by irreducible polynomials. We also show that
randomized Karp-Rabin hash families are not pairwise independent.
TripleSpin - a generic compact paradigm for fast machine learning computations
Authors: Krzysztof Choromanski, Francois Fagan, Cedric Gouy-Pailler, Anne Morvan, Tamas Sarlos, Jamal Atif
Date: 2016-05-29 00:00:00
Tasks: BIG-bench Machine Learning, Quantization

We present a generic compact computational framework relying on structured
random matrices that can be applied to speed up several machine learning
algorithms with almost no loss of accuracy. The applications include new fast
LSH-based algorithms, efficient kernel computations via random feature maps,
convex optimization algorithms, quantization techniques and many more. Certain
models of the presented paradigm are even more compressible since they apply
only bit matrices. This makes them suitable for deploying on mobile devices.
All our findings come with strong theoretical guarantees. In particular, as a
byproduct of the presented techniques and by using relatively new
Berry-Esseen-type CLT for random vectors, we give the first theoretical
guarantees for one of the most efficient existing LSH algorithms based on the
$\textbf{HD}_{3}\textbf{HD}_{2}\textbf{HD}_{1}$ structured matrix ("Practical
and Optimal LSH for Angular Distance"). These guarantees as well as theoretical
results for other aforementioned applications follow from the same general
theoretical principle that we present in the paper. Our structured family
contains as special cases all previously considered structured schemes,
including the recently introduced $P$-model. Experimental evaluation confirms
the accuracy and efficiency of TripleSpin matrices.
Unsupervised classification of children's bodies using currents
Authors: Sonia Barahona, Ximo Gual-Arnau, Maria Victoria Ibáñez, Amelia Simó
Date: 2016-06-06 00:00:00
Tasks: Classification, General Classification, Object

Object classification according to their shape and size is of key importance
in many scientific fields. This work focuses on the case where the size and
shape of an object is characterized by a current}. A current is a mathematical
object which has been proved relevant to the modeling of geometrical data, like
submanifolds, through integration of vector fields along them. As a consequence
of the choice of a vector-valued Reproducing Kernel Hilbert Space (RKHS) as a
test space for integrating manifolds, it is possible to consider that shapes
are embedded in this Hilbert Space. A vector-valued RKHS is a Hilbert space of
vector fields; therefore, it is possible to compute a mean of shapes, or to
calculate a distance between two manifolds. This embedding enables us to
consider size-and-shape classification algorithms.
  These algorithms are applied to a 3D database obtained from an anthropometric
survey of the Spanish child population with a potential application to online
sales of children's wear.
Expected Similarity Estimation for Large-Scale Batch and Streaming Anomaly Detection
Authors: Markus Schneider, Wolfgang Ertel, Fabio Ramos
Date: 2016-01-25 00:00:00
Tasks: Anomaly Detection, Incremental Learning

We present a novel algorithm for anomaly detection on very large datasets and
data streams. The method, named EXPected Similarity Estimation (EXPoSE), is
kernel-based and able to efficiently compute the similarity between new data
points and the distribution of regular data. The estimator is formulated as an
inner product with a reproducing kernel Hilbert space embedding and makes no
assumption about the type or shape of the underlying data distribution. We show
that offline (batch) learning with EXPoSE can be done in linear time and online
(incremental) learning takes constant time per instance and model update.
Furthermore, EXPoSE can make predictions in constant time, while it requires
only constant memory. In addition, we propose different methodologies for
concept drift adaptation on evolving data streams. On several real datasets we
demonstrate that our approach can compete with state of the art algorithms for
anomaly detection while being an order of magnitude faster than most other
approaches.
Proof nets for the Displacement calculus
Authors: Richard Moot
Date: 2016-06-06 00:00:00
Tasks: Translation

We present a proof net calculus for the Displacement calculus and show its
correctness. This is the first proof net calculus which models the Displacement
calculus directly and not by some sort of translation into another formalism.
The proof net calculus opens up new possibilities for parsing and proof search
with the Displacement calculus.