pwc-archive/papers-with-abstracts huggingface dataset:
Machine Translation Evaluation with Neural Networks
Authors: Francisco Guzmán, Shafiq R. Joty, Lluís Màrquez, Preslav Nakov
Date: 2017-10-05 00:00:00
Tasks: Machine Translation, Sentence, Translation

We present a framework for machine translation evaluation using neural
networks in a pairwise setting, where the goal is to select the better
translation from a pair of hypotheses, given the reference translation. In this
framework, lexical, syntactic and semantic information from the reference and
the two hypotheses is embedded into compact distributed vector representations,
and fed into a multi-layer neural network that models nonlinear interactions
between each of the hypotheses and the reference, as well as between the two
hypotheses. We experiment with the benchmark datasets from the WMT Metrics
shared task, on which we obtain the best results published so far, with the
basic network configuration. We also perform a series of experiments to analyze
and understand the contribution of the different components of the network. We
evaluate variants and extensions, including fine-tuning of the semantic
embeddings, and sentence-based representations modeled with convolutional and
recurrent neural networks. In summary, the proposed framework is flexible and
generalizable, allows for efficient learning and scoring, and provides an MT
evaluation metric that correlates with human judgments, and is on par with the
state of the art.
Neural network an1alysis of sleep stages enables efficient diagnosis of narcolepsy
Authors: Jens B. Stephansen, Alexander N. Olesen, Mads Olsen, Aditya Ambati, Eileen B. Leary, Hyatt E. Moore, Oscar Carrillo, Ling Lin, Fang Han, Han Yan, Yun L. Sun, Yves Dauvilliers, Sabine Scholz, Lucie Barateau, Birgit Hogl, Ambra Stefani, Seung Chul Hong, Tae Won Kim, Fabio Pizza, Giuseppe Plazzi, Stefano Vandi, Elena Antelmi, Dimitri Perrin, Samuel T. Kuna, Paula K. Schweitzer, Clete Kushida, Paul E. Peppard, Helge B. D. Sorensen, Poul Jennum, Emmanuel Mignot
Date: 2017-10-05 00:00:00
Tasks: Specificity

Analysis of sleep for the diagnosis of sleep disorders such as Type-1
Narcolepsy (T1N) currently requires visual inspection of polysomnography
records by trained scoring technicians. Here, we used neural networks in
approximately 3,000 normal and abnormal sleep recordings to automate sleep
stage scoring, producing a hypnodensity graph - a probability distribution
conveying more information than classical hypnograms. Accuracy of sleep stage
scoring was validated in 70 subjects assessed by six scorers. The best model
performed better than any individual scorer (87% versus consensus). It also
reliably scores sleep down to 5 instead of 30 second scoring epochs. A T1N
marker based on unusual sleep-stage overlaps achieved a specificity of 96% and
a sensitivity of 91%, validated in independent datasets. Addition of
HLA-DQB1*06:02 typing increased specificity to 99%. Our method can reduce time
spent in sleep clinics and automates T1N diagnosis. It also opens the
possibility of diagnosing T1N using home sleep studies.
Nonparametric Bayesian Negative Binomial Factor Analysis
Authors: Mingyuan Zhou
Date: 2016-04-25 00:00:00

A common approach to analyze a covariate-sample count matrix, an element of
which represents how many times a covariate appears in a sample, is to
factorize it under the Poisson likelihood. We show its limitation in capturing
the tendency for a covariate present in a sample to both repeat itself and
excite related ones. To address this limitation, we construct negative binomial
factor analysis (NBFA) to factorize the matrix under the negative binomial
likelihood, and relate it to a Dirichlet-multinomial distribution based
mixed-membership model. To support countably infinite factors, we propose the
hierarchical gamma-negative binomial process. By exploiting newly proved
connections between discrete distributions, we construct two blocked and a
collapsed Gibbs sampler that all adaptively truncate their number of factors,
and demonstrate that the blocked Gibbs sampler developed under a compound
Poisson representation converges fast and has low computational complexity.
Example results show that NBFA has a distinct mechanism in adjusting its number
of inferred factors according to the sample lengths, and provides clear
advantages in parsimonious representation, predictive power, and computational
complexity over previously proposed discrete latent variable models, which
either completely ignore burstiness, or model only the burstiness of the
covariates but not that of the factors.
Online Photometric Calibration for Auto Exposure Video for Realtime Visual Odometry and SLAM
Authors: Paul Bergmann, Rui Wang, Daniel Cremers
Date: 2017-10-05 00:00:00
Tasks: Camera Calibration, Visual Odometry

Recent direct visual odometry and SLAM algorithms have demonstrated
impressive levels of precision. However, they require a photometric camera
calibration in order to achieve competitive results. Hence, the respective
algorithm cannot be directly applied to an off-the-shelf-camera or to a video
sequence acquired with an unknown camera. In this work we propose a method for
online photometric calibration which enables to process auto exposure videos
with visual odometry precisions that are on par with those of photometrically
calibrated videos. Our algorithm recovers the exposure times of consecutive
frames, the camera response function, and the attenuation factors of the sensor
irradiance due to vignetting. Gain robust KLT feature tracks are used to obtain
scene point correspondences as input to a nonlinear optimization framework. We
show that our approach can reliably calibrate arbitrary video sequences by
evaluating it on datasets for which full photometric ground truth is available.
We further show that our calibration can improve the performance of a
state-of-the-art direct visual odometry method that works solely on pixel
intensities, calibrating for photometric parameters in an online fashion in
realtime.
Probabilistic Synchronous Parallel
Authors: Liang Wang, Ben Catterall, Richard Mortier
Date: 2017-09-22 00:00:00

Most machine learning and deep neural network algorithms rely on certain
iterative algorithms to optimise their utility/cost functions, e.g. Stochastic
Gradient Descent. In distributed learning, the networked nodes have to work
collaboratively to update the model parameters, and the way how they proceed is
referred to as synchronous parallel design (or barrier control). Synchronous
parallel protocol is the building block of any distributed learning framework,
and its design has direct impact on the performance and scalability of the
system.
  In this paper, we propose a new barrier control technique - Probabilistic
Synchronous Parallel (PSP). Com- paring to the previous Bulk Synchronous
Parallel (BSP), Stale Synchronous Parallel (SSP), and (Asynchronous Parallel)
ASP, the proposed solution e ectively improves both the convergence speed and
the scalability of the SGD algorithm by introducing a sampling primitive into
the system. Moreover, we also show that the sampling primitive can be applied
atop of the existing barrier control mechanisms to derive fully distributed
PSP-based synchronous parallel.
  We not only provide a thorough theoretical analysis1 on the convergence of
PSP-based SGD algorithm, but also implement a full-featured distributed
learning framework called Actor and perform intensive evaluation atop of it.
On the Effective Use of Pretraining for Natural Language Inference
Authors: Ignacio Cases, Minh-Thang Luong, Christopher Potts
Date: 2017-10-05 00:00:00
Tasks: Natural Language Inference

Neural networks have excelled at many NLP tasks, but there remain open
questions about the performance of pretrained distributed word representations
and their interaction with weight initialization and other hyperparameters. We
address these questions empirically using attention-based sequence-to-sequence
models for natural language inference (NLI). Specifically, we compare three
types of embeddings: random, pretrained (GloVe, word2vec), and retrofitted
(pretrained plus WordNet information). We show that pretrained embeddings
outperform both random and retrofitted ones in a large NLI corpus. Further
experiments on more controlled data sets shed light on the contexts for which
retrofitted embeddings can be useful. We also explore two principled approaches
to initializing the rest of the model parameters, Gaussian and orthogonal,
showing that the latter yields gains of up to 2.9% in the NLI task.
A Unifying Framework for Gaussian Process Pseudo-Point Approximations using Power Expectation Propagation
Authors: Thang D. Bui, Josiah Yan, Richard E. Turner
Date: 2016-05-23 00:00:00
Tasks: Gaussian Processes

Gaussian processes (GPs) are flexible distributions over functions that
enable high-level assumptions about unknown functions to be encoded in a
parsimonious, flexible and general way. Although elegant, the application of
GPs is limited by computational and analytical intractabilities that arise when
data are sufficiently numerous or when employing non-Gaussian models.
Consequently, a wealth of GP approximation schemes have been developed over the
last 15 years to address these key limitations. Many of these schemes employ a
small set of pseudo data points to summarise the actual data. In this paper, we
develop a new pseudo-point approximation framework using Power Expectation
Propagation (Power EP) that unifies a large number of these pseudo-point
approximations. Unlike much of the previous venerable work in this area, the
new framework is built on standard methods for approximate inference
(variational free-energy, EP and Power EP methods) rather than employing
approximations to the probabilistic generative model itself. In this way, all
of approximation is performed at `inference time' rather than at `modelling
time' resolving awkward philosophical and empirical questions that trouble
previous approaches. Crucially, we demonstrate that the new framework includes
new pseudo-point approximation methods that outperform current approaches on
regression and classification tasks.
Integrating Boundary and Center Correlation Filters for Visual Tracking with Aspect Ratio Variation
Authors: Feng Li, Yingjie Yao, Peihua Li, David Zhang, WangMeng Zuo, Ming-Hsuan Yang
Date: 2017-10-05 00:00:00
Tasks: Visual Tracking

The aspect ratio variation frequently appears in visual tracking and has a
severe influence on performance. Although many correlation filter (CF)-based
trackers have also been suggested for scale adaptive tracking, few studies have
been given to handle the aspect ratio variation for CF trackers. In this paper,
we make the first attempt to address this issue by introducing a family of 1D
boundary CFs to localize the left, right, top, and bottom boundaries in videos.
This allows us cope with the aspect ratio variation flexibly during tracking.
Specifically, we present a novel tracking model to integrate 1D Boundary and 2D
Center CFs (IBCCF) where boundary and center filters are enforced by a
near-orthogonality regularization term. To optimize our IBCCF model, we develop
an alternating direction method of multipliers. Experiments on several datasets
show that IBCCF can effectively handle aspect ratio variation, and achieves
state-of-the-art performance in terms of accuracy and robustness.
Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensor
Authors: Franziska Mueller, Dushyant Mehta, Oleksandr Sotnychenko, Srinath Sridhar, Dan Casas, Christian Theobalt
Date: 2017-04-07 00:00:00
Proceeding: ICCV 2017 10
Tasks: Hand Pose Estimation, Pose Estimation, Pose Tracking, Position

We present an approach for real-time, robust and accurate hand pose
estimation from moving egocentric RGB-D cameras in cluttered real environments.
Existing methods typically fail for hand-object interactions in cluttered
scenes imaged from egocentric viewpoints, common for virtual or augmented
reality applications. Our approach uses two subsequently applied Convolutional
Neural Networks (CNNs) to localize the hand and regress 3D joint locations.
Hand localization is achieved by using a CNN to estimate the 2D position of the
hand center in the input, even in the presence of clutter and occlusions. The
localized hand position, together with the corresponding input depth value, is
used to generate a normalized cropped image that is fed into a second CNN to
regress relative 3D hand joint locations in real time. For added accuracy,
robustness and temporal stability, we refine the pose estimates using a
kinematic pose tracking energy. To train the CNNs, we introduce a new
photorealistic dataset that uses a merged reality approach to capture and
synthesize large amounts of annotated data of natural hand interaction in
cluttered scenes. Through quantitative and qualitative evaluation, we show that
our method is robust to self-occlusion and occlusions by objects, particularly
in moving egocentric perspectives.
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond
Authors: Levent Sagun, Leon Bottou, Yann Lecun
Date: 2016-11-22 00:00:00
Tasks: Deep Learning

We look at the eigenvalues of the Hessian of a loss function before and after
training. The eigenvalue distribution is seen to be composed of two parts, the
bulk which is concentrated around zero, and the edges which are scattered away
from zero. We present empirical evidence for the bulk indicating how
over-parametrized the system is, and for the edges that depend on the input
data.
Alternating Iteratively Reweighted Minimization Algorithms for Low-Rank Matrix Factorization
Authors: Paris V. Giampouras, Athanasios A. Rontogiannis, Konstantinos D. Koutroumbas
Date: 2017-10-05 00:00:00
Tasks: Denoising, Matrix Completion

Nowadays, the availability of large-scale data in disparate application
domains urges the deployment of sophisticated tools for extracting valuable
knowledge out of this huge bulk of information. In that vein, low-rank
representations (LRRs) which seek low-dimensional embeddings of data have
naturally appeared. In an effort to reduce computational complexity and improve
estimation performance, LRR has been viewed via a matrix factorization (MF)
perspective. Recently, low-rank MF (LRMF) approaches have been proposed for
tackling the inherent weakness of MF i.e., the unawareness of the dimension of
the low-dimensional space where data reside. Herein, inspired by the merits of
iterative reweighted schemes for rank minimization, we come up with a generic
low-rank promoting regularization function. Then, focusing on a specific
instance of it, we propose a regularizer that imposes column-sparsity jointly
on the two matrix factors that result from MF, thus promoting low-rankness on
the optimization problem. The problems of denoising, matrix completion and
non-negative matrix factorization (NMF) are redefined according to the new LRMF
formulation and solved via efficient Newton-type algorithms with proven
theoretical guarantees as to their convergence and rates of convergence to
stationary points. The effectiveness of the proposed algorithms is verified in
diverse simulated and real data experiments.
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Authors: Alex Kendall, Yarin Gal
Date: 2017-03-15 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Deep Learning, Depth Estimation, regression, Segmentation, Semantic Segmentation

There are two major types of uncertainty one can model. Aleatoric uncertainty
captures noise inherent in the observations. On the other hand, epistemic
uncertainty accounts for uncertainty in the model -- uncertainty which can be
explained away given enough data. Traditionally it has been difficult to model
epistemic uncertainty in computer vision, but with new Bayesian deep learning
tools this is now possible. We study the benefits of modeling epistemic vs.
aleatoric uncertainty in Bayesian deep learning models for vision tasks. For
this we present a Bayesian deep learning framework combining input-dependent
aleatoric uncertainty together with epistemic uncertainty. We study models
under the framework with per-pixel semantic segmentation and depth regression
tasks. Further, our explicit uncertainty formulation leads to new loss
functions for these tasks, which can be interpreted as learned attenuation.
This makes the loss more robust to noisy data, also giving new state-of-the-art
results on segmentation and depth regression benchmarks.
Towards lightweight convolutional neural networks for object detection
Authors: Dmitriy Anisimov, Tatiana Khanova
Date: 2017-07-05 00:00:00
Tasks: CPU, Object, object-detection, Object Detection, vehicle detection

We propose model with larger spatial size of feature maps and evaluate it on
object detection task. With the goal to choose the best feature extraction
network for our model we compare several popular lightweight networks. After
that we conduct a set of experiments with channels reduction algorithms in
order to accelerate execution. Our vehicle detection models are accurate, fast
and therefore suit for embedded visual applications. With only 1.5 GFLOPs our
best model gives 93.39 AP on validation subset of challenging DETRAC dataset.
The smallest of our models is the first to achieve real-time inference speed on
CPU with reasonable accuracy drop to 91.43 AP.
Learning rates for classification with Gaussian kernels
Authors: Shao-Bo Lin, Jinshan Zeng, Xiangyu Chang
Date: 2017-02-28 00:00:00
Tasks: Binary Classification, Classification, General Classification, regression

This paper aims at refined error analysis for binary classification using
support vector machine (SVM) with Gaussian kernel and convex loss. Our first
result shows that for some loss functions such as the truncated quadratic loss
and quadratic loss, SVM with Gaussian kernel can reach the almost optimal
learning rate, provided the regression function is smooth. Our second result
shows that, for a large number of loss functions, under some Tsybakov noise
assumption, if the regression function is infinitely smooth, then SVM with
Gaussian kernel can achieve the learning rate of order $m^{-1}$, where $m$ is
the number of samples.
Machine Learning Based Detection of Clickbait Posts in Social Media
Authors: Xinyue Cao, Thai Le, Jason, Zhang
Date: 2017-10-05 00:00:00
Tasks: BIG-bench Machine Learning, Clickbait Detection

Clickbait (headlines) make use of misleading titles that hide critical
information from or exaggerate the content on the landing target pages to
entice clicks. As clickbaits often use eye-catching wording to attract viewers,
target contents are often of low quality. Clickbaits are especially widespread
on social media such as Twitter, adversely impacting user experience by causing
immense dissatisfaction. Hence, it has become increasingly important to put
forward a widely applicable approach to identify and detect clickbaits. In this
paper, we make use of a dataset from the clickbait challenge 2017
(clickbait-challenge.com) comprising of over 21,000 headlines/titles, each of
which is annotated by at least five judgments from crowdsourcing on how
clickbait it is. We attempt to build an effective computational clickbait
detection model on this dataset. We first considered a total of 331 features,
filtered out many features to avoid overfitting and improve the running time of
learning, and eventually selected the 60 most important features for our final
model. Using these features, Random Forest Regression achieved the following
results: MSE=0.035 MSE, Accuracy=0.82, and F1-sore=0.61 on the clickbait class.
Size Matters: Cardinality-Constrained Clustering and Outlier Detection via Conic Optimization
Authors: Napat Rujeerapaiboon, Kilian Schindler, Daniel Kuhn, Wolfram Wiesemann
Date: 2017-05-22 00:00:00
Tasks: Clustering, Constrained Clustering, Outlier Detection

Plain vanilla K-means clustering has proven to be successful in practice, yet
it suffers from outlier sensitivity and may produce highly unbalanced clusters.
To mitigate both shortcomings, we formulate a joint outlier detection and
clustering problem, which assigns a prescribed number of datapoints to an
auxiliary outlier cluster and performs cardinality-constrained K-means
clustering on the residual dataset, treating the cluster cardinalities as a
given input. We cast this problem as a mixed-integer linear program (MILP) that
admits tractable semidefinite and linear programming relaxations. We propose
deterministic rounding schemes that transform the relaxed solutions to feasible
solutions for the MILP. We also prove that these solutions are optimal in the
MILP if a cluster separation condition holds.
Improving Implicit Semantic Role Labeling by Predicting Semantic Frame Arguments
Authors: Quynh Ngoc Thi Do, Steven Bethard, Marie-Francine Moens
Date: 2017-04-10 00:00:00
Proceeding: IJCNLP 2017 11
Tasks: Common Sense Reasoning, Semantic Role Labeling

Implicit semantic role labeling (iSRL) is the task of predicting the semantic
roles of a predicate that do not appear as explicit arguments, but rather
regard common sense knowledge or are mentioned earlier in the discourse. We
introduce an approach to iSRL based on a predictive recurrent neural semantic
frame model (PRNSFM) that uses a large unannotated corpus to learn the
probability of a sequence of semantic arguments given a predicate. We leverage
the sequence probabilities predicted by the PRNSFM to estimate selectional
preferences for predicates and their arguments. On the NomBank iSRL test set,
our approach improves state-of-the-art performance on implicit semantic role
labeling with less reliance than prior work on manually constructed language
resources.
Learning RBM with a DC programming Approach
Authors: Vidyadhar Upadhya, P. S. Sastry
Date: 2017-09-21 00:00:00

By exploiting the property that the RBM log-likelihood function is the
difference of convex functions, we formulate a stochastic variant of the
difference of convex functions (DC) programming to minimize the negative
log-likelihood. Interestingly, the traditional contrastive divergence algorithm
is a special case of the above formulation and the hyperparameters of the two
algorithms can be chosen such that the amount of computation per mini-batch is
identical. We show that for a given computational budget the proposed algorithm
almost always reaches a higher log-likelihood more rapidly, compared to the
standard contrastive divergence algorithm. Further, we modify this algorithm to
use the centered gradients and show that it is more efficient and effective
compared to the standard centered gradient algorithm on benchmark datasets.
Semantic speech retrieval with a visually grounded model of untranscribed speech
Authors: Herman Kamper, Gregory Shakhnarovich, Karen Livescu
Date: 2017-10-05 00:00:00
Tasks: Language Acquisition, Retrieval

There is growing interest in models that can learn from unlabelled speech
paired with visual context. This setting is relevant for low-resource speech
processing, robotics, and human language acquisition research. Here we study
how a visually grounded speech model, trained on images of scenes paired with
spoken captions, captures aspects of semantics. We use an external image tagger
to generate soft text labels from images, which serve as targets for a neural
model that maps untranscribed speech to (semantic) keyword labels. We introduce
a newly collected data set of human semantic relevance judgements and an
associated task, semantic speech retrieval, where the goal is to search for
spoken utterances that are semantically relevant to a given text query. Without
seeing any text, the model trained on parallel speech and images achieves a
precision of almost 60% on its top ten semantic retrievals. Compared to a
supervised model trained on transcriptions, our model matches human judgements
better by some measures, especially in retrieving non-verbatim semantic
matches. We perform an extensive analysis of the model and its resulting
representations.