pwc-archive/papers-with-abstracts huggingface dataset:
Towards Neural Machine Translation with Partially Aligned Corpora
Authors: Yining Wang, Yang Zhao, Jiajun Zhang, Cheng-qing Zong, Zhengshan Xue
Date: 2017-11-03 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Machine Translation, NMT, Sentence, Translation

While neural machine translation (NMT) has become the new paradigm, the
parameter optimization requires large-scale parallel data which is scarce in
many domains and language pairs. In this paper, we address a new translation
scenario in which there only exists monolingual corpora and phrase pairs. We
propose a new method towards translation with partially aligned sentence pairs
which are derived from the phrase pairs and monolingual corpora. To make full
use of the partially aligned corpora, we adapt the conventional NMT training
method in two aspects. On one hand, different generation strategies are
designed for aligned and unaligned target words. On the other hand, a different
objective function is designed to model the partially aligned parts. The
experiments demonstrate that our method can achieve a relatively good result in
such a translation scenario, and tiny bitexts can boost translation quality to
a large extent.
Toward Robustness against Label Noise in Training Deep Discriminative Neural Networks
Authors: Arash Vahdat
Date: 2017-05-31 00:00:00
Proceeding: NeurIPS 2017 12

Collecting large training datasets, annotated with high-quality labels, is
costly and time-consuming. This paper proposes a novel framework for training
deep convolutional neural networks from noisy labeled datasets that can be
obtained cheaply. The problem is formulated using an undirected graphical model
that represents the relationship between noisy and clean labels, trained in a
semi-supervised setting. In our formulation, the inference over latent clean
labels is tractable and is regularized during training using auxiliary sources
of information. The proposed model is applied to the image labeling problem and
is shown to be effective in labeling unseen images as well as reducing label
noise in training on CIFAR-10 and MS COCO datasets.
The neighborhood lattice for encoding partial correlations in a Hilbert space
Authors: Arash A. Amini, Bryon Aragam, Qing Zhou
Date: 2017-11-03 00:00:00
Tasks: Dimensionality Reduction, regression

Neighborhood regression has been a successful approach in graphical and
structural equation modeling, with applications to learning undirected and
directed graphical models. We extend these ideas by defining and studying an
algebraic structure called the neighborhood lattice based on a generalized
notion of neighborhood regression. We show that this algebraic structure has
the potential to provide an economic encoding of all conditional independence
statements in a Gaussian distribution (or conditional uncorrelatedness in
general), even in the cases where no graphical model exists that could
"perfectly" encode all such statements. We study the computational complexity
of computing these structures and show that under a sparsity assumption, they
can be computed in polynomial time, even in the absence of the assumption of
perfectness to a graph. On the other hand, assuming perfectness, we show how
these neighborhood lattices may be "graphically" computed using the separation
properties of the so-called partial correlation graph. We also draw connections
with directed acyclic graphical models and Bayesian networks. We derive these
results using an abstract generalization of partial uncorrelatedness, called
partial orthogonality, which allows us to use algebraic properties of
projection operators on Hilbert spaces to significantly simplify and extend
existing ideas and arguments. Consequently, our results apply to a wide range
of random objects and data structures, such as random vectors, data matrices,
and functions.
Analysis of Biased Stochastic Gradient Descent Using Sequential Semidefinite Programs
Authors: Bin Hu, Peter Seiler, Laurent Lessard
Date: 2017-11-03 00:00:00

We present a convergence rate analysis for biased stochastic gradient descent (SGD), where individual gradient updates are corrupted by computation errors. We develop stochastic quadratic constraints to formulate a small linear matrix inequality (LMI) whose feasible points lead to convergence bounds of biased SGD. Based on this LMI condition, we develop a sequential minimization approach to analyze the intricate trade-offs that couple stepsize selection, convergence rate, optimization accuracy, and robustness to gradient inaccuracy. We also provide feasible points for this LMI and obtain theoretical formulas that quantify the convergence properties of biased SGD under various assumptions on the loss functions.
From which world is your graph?
Authors: Cheng Li, Felix Wong, Zhenming Liu, Varun Kanade
Date: 2017-11-03 00:00:00
Tasks: Dimensionality Reduction, Position

Discovering statistical structure from links is a fundamental problem in the
analysis of social networks. Choosing a misspecified model, or equivalently, an
incorrect inference algorithm will result in an invalid analysis or even
falsely uncover patterns that are in fact artifacts of the model. This work
focuses on unifying two of the most widely used link-formation models: the
stochastic blockmodel (SBM) and the small world (or latent space) model (SWM).
Integrating techniques from kernel learning, spectral graph theory, and
nonlinear dimensionality reduction, we develop the first statistically sound
polynomial-time algorithm to discover latent patterns in sparse graphs for both
models. When the network comes from an SBM, the algorithm outputs a block
structure. When it is from an SWM, the algorithm outputs estimates of each
node's latent position.
Learning One-hidden-layer Neural Networks with Landscape Design
Authors: Rong Ge, Jason D. Lee, Tengyu Ma
Date: 2017-11-01 00:00:00
Proceeding: ICLR 2018 1

We consider the problem of learning a one-hidden-layer neural network: we
assume the input $x\in \mathbb{R}^d$ is from Gaussian distribution and the
label $y = a^\top \sigma(Bx) + \xi$, where $a$ is a nonnegative vector in
$\mathbb{R}^m$ with $m\le d$, $B\in \mathbb{R}^{m\times d}$ is a full-rank
weight matrix, and $\xi$ is a noise vector. We first give an analytic formula
for the population risk of the standard squared loss and demonstrate that it
implicitly attempts to decompose a sequence of low-rank tensors simultaneously.
  Inspired by the formula, we design a non-convex objective function $G(\cdot)$
whose landscape is guaranteed to have the following properties: 1. All local
minima of $G$ are also global minima.
  2. All global minima of $G$ correspond to the ground truth parameters.
  3. The value and gradient of $G$ can be estimated using samples.
  With these properties, stochastic gradient descent on $G$ provably converges
to the global minimum and learn the ground-truth parameters. We also prove
finite sample complexity result and validate the results by simulations.
Rate Optimal Estimation and Confidence Intervals for High-dimensional Regression with Missing Covariates
Authors: Yining Wang, Jialei Wang, Sivaraman Balakrishnan, Aarti Singh
Date: 2017-02-09 00:00:00
Tasks: Missing Values, regression

Although a majority of the theoretical literature in high-dimensional
statistics has focused on settings which involve fully-observed data, settings
with missing values and corruptions are common in practice. We consider the
problems of estimation and of constructing component-wise confidence intervals
in a sparse high-dimensional linear regression model when some covariates of
the design matrix are missing completely at random. We analyze a variant of the
Dantzig selector [9] for estimating the regression model and we use a
de-biasing argument to construct component-wise confidence intervals. Our first
main result is to establish upper bounds on the estimation error as a function
of the model parameters (the sparsity level s, the expected fraction of
observed covariates $\rho_*$, and a measure of the signal strength
$\|\beta^*\|_2$). We find that even in an idealized setting where the
covariates are assumed to be missing completely at random, somewhat
surprisingly and in contrast to the fully-observed setting, there is a
dichotomy in the dependence on model parameters and much faster rates are
obtained if the covariance matrix of the random design is known. To study this
issue further, our second main contribution is to provide lower bounds on the
estimation error showing that this discrepancy in rates is unavoidable in a
minimax sense. We then consider the problem of high-dimensional inference in
the presence of missing data. We construct and analyze confidence intervals
using a de-biased estimator. In the presence of missing data, inference is
complicated by the fact that the de-biasing matrix is correlated with the pilot
estimator and this necessitates the design of a new estimator and a novel
analysis. We also complement our mathematical study with extensive simulations
on synthetic and semi-synthetic data that show the accuracy of our asymptotic
predictions for finite sample sizes.
Corrupt Bandits for Preserving Local Privacy
Authors: Pratik Gajane, Tanguy Urvoy, Emilie Kaufmann
Date: 2017-08-16 00:00:00
Tasks: Recommendation Systems

We study a variant of the stochastic multi-armed bandit (MAB) problem in
which the rewards are corrupted. In this framework, motivated by privacy
preservation in online recommender systems, the goal is to maximize the sum of
the (unobserved) rewards, based on the observation of transformation of these
rewards through a stochastic corruption process with known parameters. We
provide a lower bound on the expected regret of any bandit algorithm in this
corrupted setting. We devise a frequentist algorithm, KLUCB-CF, and a Bayesian
algorithm, TS-CF and give upper bounds on their regret. We also provide the
appropriate corruption parameters to guarantee a desired level of local privacy
and analyze how this impacts the regret. Finally, we present some experimental
results that confirm our analysis.
Counterfactual Conditionals in Quantified Modal Logic
Authors: Naveen Sundar Govindarajulu, Selmer Bringsjord
Date: 2017-10-11 00:00:00
Tasks: counterfactual

We present a novel formalization of counterfactual conditionals in a
quantified modal logic. Counterfactual conditionals play a vital role in
ethical and moral reasoning. Prior work has shown that moral reasoning systems
(and more generally, theory-of-mind reasoning systems) should be at least as
expressive as first-order (quantified) modal logic (QML) to be well-behaved.
While existing work on moral reasoning has focused on counterfactual-free QML
moral reasoning, we present a fully specified and implemented formal system
that includes counterfactual conditionals. We validate our model with two
projects. In the first project, we demonstrate that our system can be used to
model a complex moral principle, the doctrine of double effect. In the second
project, we use the system to build a data-set with true and false
counterfactuals as licensed by our theory, which we believe can be useful for
other researchers. This project also shows that our model can be
computationally feasible.
Running Time Analysis of the (1+1)-EA for OneMax and LeadingOnes under Bit-wise Noise
Authors: Chao Qian, Chao Bian, Wu Jiang, Ke Tang
Date: 2017-11-02 00:00:00
Tasks: Evolutionary Algorithms

In many real-world optimization problems, the objective function evaluation is subject to noise, and we cannot obtain the exact objective value. Evolutionary algorithms (EAs), a type of general-purpose randomized optimization algorithm, have been shown to be able to solve noisy optimization problems well. However, previous theoretical analyses of EAs mainly focused on noise-free optimization, which makes the theoretical understanding largely insufficient for the noisy case. Meanwhile, the few existing theoretical studies under noise often considered the one-bit noise model, which flips a randomly chosen bit of a solution before evaluation; while in many realistic applications, several bits of a solution can be changed simultaneously. In this paper, we study a natural extension of one-bit noise, the bit-wise noise model, which independently flips each bit of a solution with some probability. We analyze the running time of the (1+1)-EA solving OneMax and LeadingOnes under bit-wise noise for the first time, and derive the ranges of the noise level for polynomial and super-polynomial running time bounds. The analysis on LeadingOnes under bit-wise noise can be easily transferred to one-bit noise, and improves the previously known results. Since our analysis discloses that the (1+1)-EA can be efficient only under low noise levels, we also study whether the sampling strategy can bring robustness to noise. We prove that using sampling can significantly increase the largest noise level allowing a polynomial running time, that is, sampling is robust to noise.
Automatic Query Image Disambiguation for Content-Based Image Retrieval
Authors: Björn Barz, Joachim Denzler
Date: 2017-11-02 00:00:00
Tasks: Content-Based Image Retrieval, Image Retrieval, Retrieval

Query images presented to content-based image retrieval systems often have
various different interpretations, making it difficult to identify the search
objective pursued by the user. We propose a technique for overcoming this
ambiguity, while keeping the amount of required user interaction at a minimum.
To achieve this, the neighborhood of the query image is divided into coherent
clusters from which the user may choose the relevant ones. A novel feedback
integration technique is then employed to re-rank the entire database with
regard to both the user feedback and the original query. We evaluate our
approach on the publicly available MIRFLICKR-25K dataset, where it leads to a
relative improvement of average precision by 23% over the baseline retrieval,
which does not distinguish between different image senses.
Learning Linear Dynamical Systems via Spectral Filtering
Authors: Elad Hazan, Karan Singh, Cyril Zhang
Date: 2017-11-02 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Time Series, Time Series Analysis

We present an efficient and practical algorithm for the online prediction of
discrete-time linear dynamical systems with a symmetric transition matrix. We
circumvent the non-convex optimization problem using improper learning:
carefully overparameterize the class of LDSs by a polylogarithmic factor, in
exchange for convexity of the loss functions. From this arises a
polynomial-time algorithm with a near-optimal regret guarantee, with an
analogous sample complexity bound for agnostic learning. Our algorithm is based
on a novel filtering technique, which may be of independent interest: we
convolve the time series with the eigenvectors of a certain Hankel matrix.
Deep Active Learning over the Long Tail
Authors: Yonatan Geifman, Ran El-Yaniv
Date: 2017-11-02 00:00:00
Tasks: Active Learning

This paper is concerned with pool-based active learning for deep neural
networks. Motivated by coreset dataset compression ideas, we present a novel
active learning algorithm that queries consecutive points from the pool using
farthest-first traversals in the space of neural activation over a
representation layer. We show consistent and overwhelming improvement in sample
complexity over passive learning (random sampling) for three datasets: MNIST,
CIFAR-10, and CIFAR-100. In addition, our algorithm outperforms the traditional
uncertainty sampling technique (obtained using softmax activations), and we
identify cases where uncertainty sampling is only slightly better than random
sampling.
A Comparison of Feature-Based and Neural Scansion of Poetry
Authors: Manex Agirrezabal, Iñaki Alegria, Mans Hulden
Date: 2017-11-02 00:00:00
Proceeding: RANLP 2017 9
Tasks: Rhythm

Automatic analysis of poetic rhythm is a challenging task that involves
linguistics, literature, and computer science. When the language to be analyzed
is known, rule-based systems or data-driven methods can be used. In this paper,
we analyze poetic rhythm in English and Spanish. We show that the
representations of data learned from character-based neural models are more
informative than the ones from hand-crafted features, and that a
Bi-LSTM+CRF-model produces state-of-the art accuracy on scansion of poetry in
two languages. Results also show that the information about whole word
structure, and not just independent syllables, is highly informative for
performing scansion.
Binary Bouncy Particle Sampler
Authors: Ari Pakman
Date: 2017-11-02 00:00:00

The Bouncy Particle Sampler is a novel rejection-free non-reversible sampler
for differentiable probability distributions over continuous variables. We
generalize the algorithm to piecewise differentiable distributions and apply it
to generic binary distributions using a piecewise differentiable augmentation.
We illustrate the new algorithm in a binary Markov Random Field example, and
compare it to binary Hamiltonian Monte Carlo. Our results suggest that binary
BPS samplers are better for easy to mix distributions.
Deep Learning and Model Predictive Control for Self-Tuning Mode-Locked Lasers
Authors: Thomas Baumeister, Steven L. Brunton, J. Nathan Kutz
Date: 2017-11-02 00:00:00
Tasks: Model Predictive Control

Self-tuning optical systems are of growing importance in technological
applications such as mode-locked fiber lasers. Such self-tuning paradigms
require {\em intelligent} algorithms capable of inferring approximate models of
the underlying physics and discovering appropriate control laws in order to
maintain robust performance for a given objective. In this work, we demonstrate
the first integration of a {\em deep learning} (DL) architecture with {\em
model predictive control} (MPC) in order to self-tune a mode-locked fiber
laser. Not only can our DL-MPC algorithmic architecture approximate the unknown
fiber birefringence, it also builds a dynamical model of the laser and
appropriate control law for maintaining robust, high-energy pulses despite a
stochastically drifting birefringence. We demonstrate the effectiveness of this
method on a fiber laser which is mode-locked by nonlinear polarization
rotation. The method advocated can be broadly applied to a variety of optical
systems that require robust controllers.
Does Phase Matter For Monaural Source Separation?
Authors: Mohit Dubey, Garrett Kenyon, Nils Carlson, Austin Thresher
Date: 2017-11-02 00:00:00

The "cocktail party" problem of fully separating multiple sources from a
single channel audio waveform remains unsolved. Current biological
understanding of neural encoding suggests that phase information is preserved
and utilized at every stage of the auditory pathway. However, current
computational approaches primarily discard phase information in order to mask
amplitude spectrograms of sound. In this paper, we seek to address whether
preserving phase information in spectral representations of sound provides
better results in monaural separation of vocals from a musical track by using a
neurally plausible sparse generative model. Our results demonstrate that
preserving phase information reduces artifacts in the separated tracks, as
quantified by the signal to artifact ratio (GSAR). Furthermore, our proposed
method achieves state-of-the-art performance for source separation, as
quantified by a mean signal to interference ratio (GSIR) of 19.46.
Hi, how can I help you?: Automating enterprise IT support help desks
Authors: Senthil Mani, Neelamadhav Gantayat, Rahul Aralikatte, Monika Gupta, Sampath Dechu, Anush Sankaran, Shreya Khare, Barry Mitchell, Hemamalini Subramanian, Hema Venkatarangan
Date: 2017-11-02 00:00:00
Tasks: Management, Natural Language Understanding, Question Answering, Retrieval

Question answering is one of the primary challenges of natural language
understanding. In realizing such a system, providing complex long answers to
questions is a challenging task as opposed to factoid answering as the former
needs context disambiguation. The different methods explored in the literature
can be broadly classified into three categories namely: 1) classification
based, 2) knowledge graph based and 3) retrieval based. Individually, none of
them address the need of an enterprise wide assistance system for an IT support
and maintenance domain. In this domain the variance of answers is large ranging
from factoid to structured operating procedures; the knowledge is present
across heterogeneous data sources like application specific documentation,
ticket management systems and any single technique for a general purpose
assistance is unable to scale for such a landscape. To address this, we have
built a cognitive platform with capabilities adopted for this domain. Further,
we have built a general purpose question answering system leveraging the
platform that can be instantiated for multiple products, technologies in the
support domain. The system uses a novel hybrid answering model that
orchestrates across a deep learning classifier, a knowledge graph based context
disambiguation module and a sophisticated bag-of-words search system. This
orchestration performs context switching for a provided question and also does
a smooth hand-off of the question to a human expert if none of the automated
techniques can provide a confident answer. This system has been deployed across
675 internal enterprise IT support and maintenance projects.
Weight-Based Variable Ordering in the Context of High-Level Consistencies
Authors: Robert J. Woodward, Berthe Y. Choueiry
Date: 2017-11-02 00:00:00
Tasks: ARC, Vocal Bursts Intensity Prediction

Dom/wdeg is one of the best performing heuristics for dynamic variable
ordering in backtrack search [Boussemart et al., 2004]. As originally defined,
this heuristic increments the weight of the constraint that causes a domain
wipeout (i.e., a dead-end) when enforcing arc consistency during search. "The
process of weighting constraints with dom/wdeg is not defined when more than
one constraint lead to a domain wipeout [Vion et al., 2011]." In this paper, we
investigate how weights should be updated in the context of two high-level
consistencies, namely, singleton (POAC) and relational consistencies (RNIC). We
propose, analyze, and empirically evaluate several strategies for updating the
weights. We statistically compare the proposed strategies and conclude with our
recommendations.