pwc-archive/papers-with-abstracts huggingface dataset:
Primal-Dual Optimization Algorithms over Riemannian Manifolds: an Iteration Complexity Analysis
Authors: Junyu Zhang, Shiqian Ma, Shuzhong Zhang
Date: 2017-10-05 00:00:00
Tasks: BIG-bench Machine Learning, Community Detection, Compressive Sensing

In this paper we study nonconvex and nonsmooth multi-block optimization over
Riemannian manifolds with coupled linear constraints. Such optimization
problems naturally arise from machine learning, statistical learning,
compressive sensing, image processing, and tensor PCA, among others. We develop
an ADMM-like primal-dual approach based on decoupled solvable subroutines such
as linearized proximal mappings. First, we introduce the optimality conditions
for the afore-mentioned optimization models. Then, the notion of
$\epsilon$-stationary solutions is introduced as a result. The main part of the
paper is to show that the proposed algorithms enjoy an iteration complexity of
$O(1/\epsilon^2)$ to reach an $\epsilon$-stationary solution. For prohibitively
large-size tensor or machine learning models, we present a sampling-based
stochastic algorithm with the same iteration complexity bound in expectation.
In case the subproblems are not analytically solvable, a feasible curvilinear
line-search variant of the algorithm based on retraction operators is proposed.
Finally, we show specifically how the algorithms can be implemented to solve a
variety of practical problems such as the NP-hard maximum bisection problem,
the $\ell_q$ regularized sparse tensor principal component analysis and the
community detection problem. Our preliminary numerical results show great
potentials of the proposed methods.
Cascaded Scene Flow Prediction using Semantic Segmentation
Authors: Zhile Ren, Deqing Sun, Jan Kautz, Erik B. Sudderth
Date: 2017-07-26 00:00:00
Tasks: 3D geometry, Autonomous Driving, General Classification, Optical Flow Estimation, Prediction, Semantic Segmentation, Superpixels

Given two consecutive frames from a pair of stereo cameras, 3D scene flow
methods simultaneously estimate the 3D geometry and motion of the observed
scene. Many existing approaches use superpixels for regularization, but may
predict inconsistent shapes and motions inside rigidly moving objects. We
instead assume that scenes consist of foreground objects rigidly moving in
front of a static background, and use semantic cues to produce pixel-accurate
scene flow estimates. Our cascaded classification framework accurately models
3D scenes by iteratively refining semantic segmentation masks, stereo
correspondences, 3D rigid motion estimates, and optical flow fields. We
evaluate our method on the challenging KITTI autonomous driving benchmark, and
show that accounting for the motion of segmented vehicles leads to
state-of-the-art performance.
Temporal Information Extraction for Question Answering Using Syntactic Dependencies in an LSTM-based Architecture
Authors: Yuanliang Meng, Anna Rumshisky, Alexey Romanov
Date: 2017-03-17 00:00:00
Proceeding: EMNLP 2017 9
Tasks: General Classification, Question Answering, Sentence, Temporal Information Extraction

In this paper, we propose to use a set of simple, uniform in architecture
LSTM-based models to recover different kinds of temporal relations from text.
Using the shortest dependency path between entities as input, the same
architecture is used to extract intra-sentence, cross-sentence, and document
creation time relations. A "double-checking" technique reverses entity pairs in
classification, boosting the recall of positive cases and reducing
misclassifications between opposite classes. An efficient pruning algorithm
resolves conflicts globally. Evaluated on QA-TempEval (SemEval2015 Task 5), our
proposed technique outperforms state-of-the-art methods by a large margin.
Stacked Structure Learning for Lifted Relational Neural Networks
Authors: Gustav Sourek, Martin Svatos, Filip Zelezny, Steven Schockaert, Ondrej Kuzelka
Date: 2017-10-05 00:00:00

Lifted Relational Neural Networks (LRNNs) describe relational domains using
weighted first-order rules which act as templates for constructing feed-forward
neural networks. While previous work has shown that using LRNNs can lead to
state-of-the-art results in various ILP tasks, these results depended on
hand-crafted rules. In this paper, we extend the framework of LRNNs with
structure learning, thus enabling a fully automated learning process. Similarly
to many ILP methods, our structure learning algorithm proceeds in an iterative
fashion by top-down searching through the hypothesis space of all possible Horn
clauses, considering the predicates that occur in the training examples as well
as invented soft concepts entailed by the best weighted rules found so far. In
the experiments, we demonstrate the ability to automatically induce useful
hierarchical soft concepts leading to deep LRNNs with a competitive predictive
power.
Video Denoising and Enhancement via Dynamic Video Layering
Authors: Han Guo, Namrata Vaswani
Date: 2017-10-05 00:00:00
Tasks: Denoising, Video Denoising

Video denoising refers to the problem of removing "noise" from a video
sequence. Here the term "noise" is used in a broad sense to refer to any
corruption or outlier or interference that is not the quantity of interest. In
this work, we develop a novel approach to video denoising that is based on the
idea that many noisy or corrupted videos can be split into three parts - the
"low-rank layer", the "sparse layer", and a small residual (which is small and
bounded). We show, using extensive experiments, that our denoising approach
outperforms the state-of-the-art denoising algorithms.
Constructing multi-modality and multi-classifier radiomics predictive models through reliable classifier fusion
Authors: Zhiguo Zhou, Zhi-Jie Zhou, Hongxia Hao, Shulong Li, Xi Chen, You Zhang, Michael Folkert, Jing Wang
Date: 2017-10-04 00:00:00

Radiomics aims to extract and analyze large numbers of quantitative features
from medical images and is highly promising in staging, diagnosing, and
predicting outcomes of cancer treatments. Nevertheless, several challenges need
to be addressed to construct an optimal radiomics predictive model. First, the
predictive performance of the model may be reduced when features extracted from
an individual imaging modality are blindly combined into a single predictive
model. Second, because many different types of classifiers are available to
construct a predictive model, selecting an optimal classifier for a particular
application is still challenging. In this work, we developed multi-modality and
multi-classifier radiomics predictive models that address the aforementioned
issues in currently available models. Specifically, a new reliable classifier
fusion strategy was proposed to optimally combine output from different
modalities and classifiers. In this strategy, modality-specific classifiers
were first trained, and an analytic evidential reasoning (ER) rule was
developed to fuse the output score from each modality to construct an optimal
predictive model. One public data set and two clinical case studies were
performed to validate model performance. The experimental results indicated
that the proposed ER rule based radiomics models outperformed the traditional
models that rely on a single classifier or simply use combined features from
different modalities.
Exploration in Feature Space for Reinforcement Learning
Authors: Suraj Narayanan Sasikumar
Date: 2017-10-05 00:00:00
Tasks: Montezuma's Revenge, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The infamous exploration-exploitation dilemma is one of the oldest and most
important problems in reinforcement learning (RL). Deliberate and effective
exploration is necessary for RL agents to succeed in most environments.
However, until very recently even very sophisticated RL algorithms employed
simple, undirected exploration strategies in large-scale RL tasks.
  We introduce a new optimistic count-based exploration algorithm for RL that
is feasible in high-dimensional MDPs. The success of RL algorithms in these
domains depends crucially on generalization from limited training experience.
Function approximation techniques enable RL agents to generalize in order to
estimate the value of unvisited states, but at present few methods have
achieved generalization about the agent's uncertainty regarding unvisited
states. We present a new method for computing a generalized state visit-count,
which allows the agent to estimate the uncertainty associated with any state.
  In contrast to existing exploration techniques, our
$\phi$-$\textit{pseudocount}$ achieves generalization by exploiting the feature
representation of the state space that is used for value function
approximation. States that have less frequently observed features are deemed
more uncertain. The resulting $\phi$-$\textit{Exploration-Bonus}$ algorithm
rewards the agent for exploring in feature space rather than in the original
state space. This method is simpler and less computationally expensive than
some previous proposals, and achieves near state-of-the-art results on
high-dimensional RL benchmarks. In particular, we report world-class results on
several notoriously difficult Atari 2600 video games, including Montezuma's
Revenge.
Porcupine Neural Networks: (Almost) All Local Optima are Global
Authors: Soheil Feizi, Hamid Javadi, Jesse Zhang, David Tse
Date: 2017-10-05 00:00:00
Tasks: All

Neural networks have been used prominently in several machine learning and
statistics applications. In general, the underlying optimization of neural
networks is non-convex which makes their performance analysis challenging. In
this paper, we take a novel approach to this problem by asking whether one can
constrain neural network weights to make its optimization landscape have good
theoretical properties while at the same time, be a good approximation for the
unconstrained one. For two-layer neural networks, we provide affirmative
answers to these questions by introducing Porcupine Neural Networks (PNNs)
whose weight vectors are constrained to lie over a finite set of lines. We show
that most local optima of PNN optimizations are global while we have a
characterization of regions where bad local optimizers may exist. Moreover, our
theoretical and empirical results suggest that an unconstrained neural network
can be approximated using a polynomially-large PNN.
Iterative Machine Learning for Precision Trajectory Tracking with Series Elastic Actuators
Authors: Nathan Banka, W. Tony Piaskowy, Joseph Garbini, Santosh Devasia
Date: 2017-10-05 00:00:00
Tasks: BIG-bench Machine Learning

When robots operate in unknown environments small errors in postions can lead
to large variations in the contact forces, especially with typical
high-impedance designs. This can potentially damage the surroundings and/or the
robot. Series elastic actuators (SEAs) are a popular way to reduce the output
impedance of a robotic arm to improve control authority over the force exerted
on the environment. However this increased control over forces with lower
impedance comes at the cost of lower positioning precision and bandwidth. This
article examines the use of an iteratively-learned feedforward command to
improve position tracking when using SEAs. Over each iteration, the output
responses of the system to the quantized inputs are used to estimate a
linearized local system models. These estimated models are obtained using a
complex-valued Gaussian Process Regression (cGPR) technique and then, used to
generate a new feedforward input command based on the previous iteration's
error. This article illustrates this iterative machine learning (IML) technique
for a two degree of freedom (2-DOF) robotic arm, and demonstrates successful
convergence of the IML approach to reduce the tracking error.
Near-Optimal Stochastic Approximation for Online Principal Component Estimation
Authors: Chris Junchi Li, Mengdi Wang, Han Liu, Tong Zhang
Date: 2016-03-16 00:00:00

Principal component analysis (PCA) has been a prominent tool for
high-dimensional data analysis. Online algorithms that estimate the principal
component by processing streaming data are of tremendous practical and
theoretical interests. Despite its rich applications, theoretical convergence
analysis remains largely open. In this paper, we cast online PCA into a
stochastic nonconvex optimization problem, and we analyze the online PCA
algorithm as a stochastic approximation iteration. The stochastic approximation
iteration processes data points incrementally and maintains a running estimate
of the principal component. We prove for the first time a nearly optimal
finite-sample error bound for the online PCA algorithm. Under the subgaussian
assumption, we show that the finite-sample error bound closely matches the
minimax information lower bound.
BPEmb: Tokenization-free Pre-trained Subword Embeddings in 275 Languages
Authors: Benjamin Heinzerling, Michael Strube
Date: 2017-10-05 00:00:00
Proceeding: LREC 2018 5
Tasks: Entity Typing, Word Embeddings

We present BPEmb, a collection of pre-trained subword unit embeddings in 275
languages, based on Byte-Pair Encoding (BPE). In an evaluation using
fine-grained entity typing as testbed, BPEmb performs competitively, and for
some languages bet- ter than alternative subword approaches, while requiring
vastly fewer resources and no tokenization. BPEmb is available at
https://github.com/bheinzerling/bpemb
A study of Thompson Sampling with Parameter h
Authors: Qiang Ha
Date: 2017-10-05 00:00:00
Tasks: Thompson Sampling

Thompson Sampling algorithm is a well known Bayesian algorithm for solving
stochastic multi-armed bandit. At each time step the algorithm chooses each arm
with probability proportional to it being the current best arm. We modify the
strategy by introducing a paramter h which alters the importance of the
probability of an arm being the current best arm. We show that the optimality
of Thompson sampling is robust to this perturbation within a range of parameter
values for two arm bandits.
Tracking Persons-of-Interest via Unsupervised Representation Adaptation
Authors: Shun Zhang, Jia-Bin Huang, Jongwoo Lim, Yihong Gong, Jinjun Wang, Narendra Ahuja, Ming-Hsuan Yang
Date: 2017-10-05 00:00:00
Tasks: Clustering, Triplet

Multi-face tracking in unconstrained videos is a challenging problem as faces
of one person often appear drastically different in multiple shots due to
significant variations in scale, pose, expression, illumination, and make-up.
Existing multi-target tracking methods often use low-level features which are
not sufficiently discriminative for identifying faces with such large
appearance variations. In this paper, we tackle this problem by learning
discriminative, video-specific face representations using convolutional neural
networks (CNNs). Unlike existing CNN-based approaches which are only trained on
large-scale face image datasets offline, we use the contextual constraints to
generate a large number of training samples for a given video, and further
adapt the pre-trained face CNN to specific videos using discovered training
samples. Using these training samples, we optimize the embedding space so that
the Euclidean distances correspond to a measure of semantic face similarity via
minimizing a triplet loss function. With the learned discriminative features,
we apply the hierarchical clustering algorithm to link tracklets across
multiple shots to generate trajectories. We extensively evaluate the proposed
algorithm on two sets of TV sitcoms and YouTube music videos, analyze the
contribution of each component, and demonstrate significant performance
improvement over existing techniques.
A Neural Comprehensive Ranker (NCR) for Open-Domain Question Answering
Authors: Bin Bi, Hao Ma
Date: 2017-09-29 00:00:00
Tasks: Open-Domain Question Answering, Passage Ranking, Question Answering, Reading Comprehension, Retrieval

This paper proposes a novel neural machine reading model for open-domain
question answering at scale. Existing machine comprehension models typically
assume that a short piece of relevant text containing answers is already
identified and given to the models, from which the models are designed to
extract answers. This assumption, however, is not realistic for building a
large-scale open-domain question answering system which requires both deep text
understanding and identifying relevant text from corpus simultaneously.
  In this paper, we introduce Neural Comprehensive Ranker (NCR) that integrates
both passage ranking and answer extraction in one single framework. A Q&A
system based on this framework allows users to issue an open-domain question
without needing to provide a piece of text that must contain the answer.
Experiments show that the unified NCR model is able to outperform the
states-of-the-art in both retrieval of relevant text and answer extraction.
DiffuserCam: Lensless Single-exposure 3D Imaging
Authors: Nick Antipa, Grace Kuo, Reinhard Heckel, Ben Mildenhall, Emrah Bostan, Ren Ng, Laura Waller
Date: 2017-10-05 00:00:00
Tasks: compressed sensing

We demonstrate a compact and easy-to-build computational camera for
single-shot 3D imaging. Our lensless system consists solely of a diffuser
placed in front of a standard image sensor. Every point within the volumetric
field-of-view projects a unique pseudorandom pattern of caustics on the sensor.
By using a physical approximation and simple calibration scheme, we solve the
large-scale inverse problem in a computationally efficient way. The caustic
patterns enable compressed sensing, which exploits sparsity in the sample to
solve for more 3D voxels than pixels on the 2D sensor. Our 3D voxel grid is
chosen to match the experimentally measured two-point optical resolution across
the field-of-view, resulting in 100 million voxels being reconstructed from a
single 1.3 megapixel image. However, the effective resolution varies
significantly with scene content. Because this effect is common to a wide range
of computational cameras, we provide new theory for analyzing resolution in
such systems.
Multiframe Scene Flow with Piecewise Rigid Motion
Authors: Vladislav Golyanik, Kihwan Kim, Robert Maier, Matthias Nießner, Didier Stricker, Jan Kautz
Date: 2017-10-05 00:00:00
Tasks: Scene Flow Estimation

We introduce a novel multiframe scene flow approach that jointly optimizes
the consistency of the patch appearances and their local rigid motions from
RGB-D image sequences. In contrast to the competing methods, we take advantage
of an oversegmentation of the reference frame and robust optimization
techniques. We formulate scene flow recovery as a global non-linear least
squares problem which is iteratively solved by a damped Gauss-Newton approach.
As a result, we obtain a qualitatively new level of accuracy in RGB-D based
scene flow estimation which can potentially run in real-time. Our method can
handle challenging cases with rigid, piecewise rigid, articulated and moderate
non-rigid motion, and does not rely on prior knowledge about the types of
motions and deformations. Extensive experiments on synthetic and real data show
that our method outperforms state-of-the-art.
Anatomical Pattern Analysis for decoding visual stimuli in human brains
Authors: Muhammad Yousefnezhad, Daoqiang Zhang
Date: 2017-10-05 00:00:00
Tasks: Binary Classification, General Classification

Background: A universal unanswered question in neuroscience and machine
learning is whether computers can decode the patterns of the human brain.
Multi-Voxels Pattern Analysis (MVPA) is a critical tool for addressing this
question. However, there are two challenges in the previous MVPA methods, which
include decreasing sparsity and noise in the extracted features and increasing
the performance of prediction.
  Methods: In overcoming mentioned challenges, this paper proposes Anatomical
Pattern Analysis (APA) for decoding visual stimuli in the human brain. This
framework develops a novel anatomical feature extraction method and a new
imbalance AdaBoost algorithm for binary classification. Further, it utilizes an
Error-Correcting Output Codes (ECOC) method for multiclass prediction. APA can
automatically detect active regions for each category of the visual stimuli.
Moreover, it enables us to combine homogeneous datasets for applying advanced
classification.
  Results and Conclusions: Experimental studies on 4 visual categories (words,
consonants, objects and scrambled photos) demonstrate that the proposed
approach achieves superior performance to state-of-the-art methods.
Learning Local Receptive Fields and their Weight Sharing Scheme on Graphs
Authors: Jean-Charles Vialatte, Vincent Gripon, Gilles Coppin
Date: 2017-06-08 00:00:00

We propose a simple and generic layer formulation that extends the properties
of convolutional layers to any domain that can be described by a graph. Namely,
we use the support of its adjacency matrix to design learnable weight sharing
filters able to exploit the underlying structure of signals in the same fashion
as for images. The proposed formulation makes it possible to learn the weights
of the filter as well as a scheme that controls how they are shared across the
graph. We perform validation experiments with image datasets and show that
these filters offer performances comparable with convolutional ones.
Learning Graphical Models from a Distributed Stream
Authors: Yu Zhang, Srikanta Tirthapura, Graham Cormode
Date: 2017-10-05 00:00:00
Tasks: Management

A current challenge for data management systems is to support the
construction and maintenance of machine learning models over data that is
large, multi-dimensional, and evolving. While systems that could support these
tasks are emerging, the need to scale to distributed, streaming data requires
new models and algorithms. In this setting, as well as computational
scalability and model accuracy, we also need to minimize the amount of
communication between distributed processors, which is the chief component of
latency. We study Bayesian networks, the workhorse of graphical models, and
present a communication-efficient method for continuously learning and
maintaining a Bayesian network model over data that is arriving as a
distributed stream partitioned across multiple processors. We show a strategy
for maintaining model parameters that leads to an exponential reduction in
communication when compared with baseline approaches to maintain the exact MLE
(maximum likelihood estimation). Meanwhile, our strategy provides similar
prediction errors for the target distribution and for classification tasks.
Reliable Clustering of Bernoulli Mixture Models
Authors: Amir Najafi, Abolfazl Motahari, Hamid R. Rabiee
Date: 2017-10-05 00:00:00
Tasks: Clustering

A Bernoulli Mixture Model (BMM) is a finite mixture of random binary vectors with independent dimensions. The problem of clustering BMM data arises in a variety of real-world applications, ranging from population genetics to activity analysis in social networks. In this paper, we analyze the clusterability of BMMs from a theoretical perspective, when the number of clusters is unknown. In particular, we stipulate a set of conditions on the sample complexity and dimension of the model in order to guarantee the Probably Approximately Correct (PAC)-clusterability of a dataset. To the best of our knowledge, these findings are the first non-asymptotic bounds on the sample complexity of learning or clustering BMMs.