pwc-archive/papers-with-abstracts huggingface dataset:
Accelerated Primal-Dual Proximal Block Coordinate Updating Methods for Constrained Convex Optimization
Authors: Yangyang Xu, Shuzhong Zhang
Date: 2017-02-17 00:00:00

Block Coordinate Update (BCU) methods enjoy low per-update computational
complexity because every time only one or a few block variables would need to
be updated among possibly a large number of blocks. They are also easily
parallelized and thus have been particularly popular for solving problems
involving large-scale dataset and/or variables. In this paper, we propose a
primal-dual BCU method for solving linearly constrained convex program in
multi-block variables. The method is an accelerated version of a primal-dual
algorithm proposed by the authors, which applies randomization in selecting
block variables to update and establishes an $O(1/t)$ convergence rate under
weak convexity assumption. We show that the rate can be accelerated to
$O(1/t^2)$ if the objective is strongly convex. In addition, if one block
variable is independent of the others in the objective, we then show that the
algorithm can be modified to achieve a linear rate of convergence. The
numerical experiments show that the accelerated method performs stably with a
single set of parameters while the original method needs to tune the parameters
for different datasets in order to achieve a comparable level of performance.
Self-Similarity Based Time Warping
Authors: Christopher J. Tralie
Date: 2017-11-20 00:00:00
Tasks: Dynamic Time Warping, Time Series, Time Series Analysis

In this work, we explore the problem of aligning two time-ordered point
clouds which are spatially transformed and re-parameterized versions of each
other. This has a diverse array of applications such as cross modal time series
synchronization (e.g. MOCAP to video) and alignment of discretized curves in
images. Most other works that address this problem attempt to jointly uncover a
spatial alignment and correspondences between the two point clouds, or to
derive local invariants to spatial transformations such as curvature before
computing correspondences. By contrast, we sidestep spatial alignment
completely by using self-similarity matrices (SSMs) as a proxy to the
time-ordered point clouds, since self-similarity matrices are blind to
isometries and respect global geometry. Our algorithm, dubbed "Isometry Blind
Dynamic Time Warping" (IBDTW), is simple and general, and we show that its
associated dissimilarity measure lower bounds the L1 Gromov-Hausdorff distance
between the two point sets when restricted to warping paths. We also present a
local, partial alignment extension of IBDTW based on the Smith Waterman
algorithm. This eliminates the need for tedious manual cropping of time series,
which is ordinarily necessary for global alignment algorithms to function
properly.
Optimistic Robust Optimization With Applications To Machine Learning
Authors: Matthew Norton, Akiko Takeda, Alexander Mafusalov
Date: 2017-11-20 00:00:00
Tasks: BIG-bench Machine Learning

Robust Optimization has traditionally taken a pessimistic, or worst-case
viewpoint of uncertainty which is motivated by a desire to find sets of optimal
policies that maintain feasibility under a variety of operating conditions. In
this paper, we explore an optimistic, or best-case view of uncertainty and show
that it can be a fruitful approach. We show that these techniques can be used
to address a wide variety of problems. First, we apply our methods in the
context of robust linear programming, providing a method for reducing
conservatism in intuitive ways that encode economically realistic modeling
assumptions. Second, we look at problems in machine learning and find that this
approach is strongly connected to the existing literature. Specifically, we
provide a new interpretation for popular sparsity inducing non-convex
regularization schemes. Additionally, we show that successful approaches for
dealing with outliers and noise can be interpreted as optimistic robust
optimization problems. Although many of the problems resulting from our
approach are non-convex, we find that DCA or DCA-like optimization approaches
can be intuitive and efficient.
Better Agnostic Clustering Via Relaxed Tensor Norms
Authors: Pravesh K. Kothari, Jacob Steinhardt
Date: 2017-11-20 00:00:00
Tasks: Clustering

We develop a new family of convex relaxations for $k$-means clustering based
on sum-of-squares norms, a relaxation of the injective tensor norm that is
efficiently computable using the Sum-of-Squares algorithm. We give an algorithm
based on this relaxation that recovers a faithful approximation to the true
means in the given data whenever the low-degree moments of the points in each
cluster have bounded sum-of-squares norms.
  We then prove a sharp upper bound on the sum-of-squares norms for moment
tensors of any distribution that satisfies the \emph{Poincare inequality}. The
Poincare inequality is a central inequality in probability theory, and a large
class of distributions satisfy it including Gaussians, product distributions,
strongly log-concave distributions, and any sum or uniformly continuous
transformation of such distributions.
  As an immediate corollary, for any $\gamma > 0$, we obtain an efficient
algorithm for learning the means of a mixture of $k$ arbitrary \Poincare
distributions in $\mathbb{R}^d$ in time $d^{O(1/\gamma)}$ so long as the means
have separation $\Omega(k^{\gamma})$. This in particular yields an algorithm
for learning Gaussian mixtures with separation $\Omega(k^{\gamma})$, thus
partially resolving an open problem of Regev and Vijayaraghavan
\citet{regev2017learning}.
  Our algorithm works even in the outlier-robust setting where an $\epsilon$
fraction of arbitrary outliers are added to the data, as long as the fraction
of outliers is smaller than the smallest cluster. We, therefore, obtain results
in the strong agnostic setting where, in addition to not knowing the
distribution family, the data itself may be arbitrarily corrupted.
SquishedNets: Squishing SqueezeNet further for edge device scenarios via deep evolutionary synthesis
Authors: Mohammad Javad Shafiee, Francis Li, Brendan Chwyl, Alexander Wong
Date: 2017-11-20 00:00:00

While deep neural networks have been shown in recent years to outperform
other machine learning methods in a wide range of applications, one of the
biggest challenges with enabling deep neural networks for widespread deployment
on edge devices such as mobile and other consumer devices is high computational
and memory requirements. Recently, there has been greater exploration into
small deep neural network architectures that are more suitable for edge
devices, with one of the most popular architectures being SqueezeNet, with an
incredibly small model size of 4.8MB. Taking further advantage of the notion
that many applications of machine learning on edge devices are often
characterized by a low number of target classes, this study explores the
utility of combining architectural modifications and an evolutionary synthesis
strategy for synthesizing even smaller deep neural architectures based on the
more recent SqueezeNet v1.1 macroarchitecture for applications with fewer
target classes. In particular, architectural modifications are first made to
SqueezeNet v1.1 to accommodate for a 10-class ImageNet-10 dataset, and then an
evolutionary synthesis strategy is leveraged to synthesize more efficient deep
neural networks based on this modified macroarchitecture. The resulting
SquishedNets possess model sizes ranging from 2.4MB to 0.95MB (~5.17X smaller
than SqueezeNet v1.1, or 253X smaller than AlexNet). Furthermore, the
SquishedNets are still able to achieve accuracies ranging from 81.2% to 77%,
and able to process at speeds of 156 images/sec to as much as 256 images/sec on
a Nvidia Jetson TX1 embedded chip. These preliminary results show that a
combination of architectural modifications and an evolutionary synthesis
strategy can be a useful tool for producing very small deep neural network
architectures that are well-suited for edge device scenarios.
One Model for the Learning of Language
Authors: Yuan Yang
Date: 2017-11-16 00:00:00
Tasks: model, Program induction

A major target of linguistics and cognitive science has been to understand
what class of learning systems can acquire the key structures of natural
language. Until recently, the computational requirements of language have been
used to argue that learning is impossible without a highly constrained
hypothesis space. Here, we describe a learning system that is maximally
unconstrained, operating over the space of all computations, and is able to
acquire several of the key structures present natural language from positive
evidence alone. The model successfully acquires regular (e.g. $(ab)^n$),
context-free (e.g. $a^n b^n$, $x x^R$), and context-sensitive (e.g.
$a^nb^nc^n$, $a^nb^mc^nd^m$, $xx$) formal languages. Our approach develops the
concept of factorized programs in Bayesian program induction in order to help
manage the complexity of representation. We show in learning, the model
predicts several phenomena empirically observed in human grammar acquisition
experiments.
Visual Reference Resolution using Attention Memory for Visual Dialog
Authors: Paul Hongsuck Seo, Andreas Lehrmann, Bohyung Han, Leonid Sigal
Date: 2017-09-23 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Parameter Prediction, Question Answering, Visual Dialog, Visual Grounding, Visual Question Answering, Visual Question Answering (VQA)

Visual dialog is a task of answering a series of inter-dependent questions
given an input image, and often requires to resolve visual references among the
questions. This problem is different from visual question answering (VQA),
which relies on spatial attention (a.k.a. visual grounding) estimated from an
image and question pair. We propose a novel attention mechanism that exploits
visual attentions in the past to resolve the current reference in the visual
dialog scenario. The proposed model is equipped with an associative attention
memory storing a sequence of previous (attention, key) pairs. From this memory,
the model retrieves the previous attention, taking into account recency, which
is most relevant for the current question, in order to resolve potentially
ambiguous references. The model then merges the retrieved attention with a
tentative one to obtain the final attention for the current question;
specifically, we use dynamic parameter prediction to combine the two attentions
conditioned on the question. Through extensive experiments on a new synthetic
visual dialog dataset, we show that our model significantly outperforms the
state-of-the-art (by ~16 % points) in situations, where visual reference
resolution plays an important role. Moreover, the proposed model achieves
superior performance (~ 2 % points improvement) in the Visual Dialog dataset,
despite having significantly fewer parameters than the baselines.
Automatic Tool Landmark Detection for Stereo Vision in Robot-Assisted Retinal Surgery
Authors: Thomas Probst, Kevis-Kokitsi Maninis, Ajad Chhatkuli, Mouloud Ourak, Emmanuel Vander Poorten, Luc van Gool
Date: 2017-09-17 00:00:00
Tasks: 3D Reconstruction, Anatomy

Computer vision and robotics are being increasingly applied in medical
interventions. Especially in interventions where extreme precision is required
they could make a difference. One such application is robot-assisted retinal
microsurgery. In recent works, such interventions are conducted under a
stereo-microscope, and with a robot-controlled surgical tool. The
complementarity of computer vision and robotics has however not yet been fully
exploited. In order to improve the robot control we are interested in 3D
reconstruction of the anatomy and in automatic tool localization using a stereo
microscope. In this paper, we solve this problem for the first time using a
single pipeline, starting from uncalibrated cameras to reach metric 3D
reconstruction and registration, in retinal microsurgery. The key ingredients
of our method are: (a) surgical tool landmark detection, and (b) 3D
reconstruction with the stereo microscope, using the detected landmarks. To
address the former, we propose a novel deep learning method that detects and
recognizes keypoints in high definition images at higher than real-time speed.
We use the detected 2D keypoints along with their corresponding 3D coordinates
obtained from the robot sensors to calibrate the stereo microscope using an
affine projection model. We design an online 3D reconstruction pipeline that
makes use of smoothness constraints and performs robot-to-camera registration.
The entire pipeline is extensively validated on open-sky porcine eye sequences.
Quantitative and qualitative results are presented for all steps.
E-PUR: An Energy-Efficient Processing Unit for Recurrent Neural Networks
Authors: Franyell Silfa, Gem Dot, Jose-Maria Arnau, Antonio Gonzalez
Date: 2017-11-20 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Image Description, Machine Translation, speech-recognition, Speech Recognition

Recurrent Neural Networks (RNNs) are a key technology for emerging
applications such as automatic speech recognition, machine translation or image
description. Long Short Term Memory (LSTM) networks are the most successful RNN
implementation, as they can learn long term dependencies to achieve high
accuracy. Unfortunately, the recurrent nature of LSTM networks significantly
constrains the amount of parallelism and, hence, multicore CPUs and many-core
GPUs exhibit poor efficiency for RNN inference. In this paper, we present
E-PUR, an energy-efficient processing unit tailored to the requirements of LSTM
computation. The main goal of E-PUR is to support large recurrent neural
networks for low-power mobile devices. E-PUR provides an efficient hardware
implementation of LSTM networks that is flexible to support diverse
applications. One of its main novelties is a technique that we call Maximizing
Weight Locality (MWL), which improves the temporal locality of the memory
accesses for fetching the synaptic weights, reducing the memory requirements by
a large extent. Our experimental results show that E-PUR achieves real-time
performance for different LSTM networks, while reducing energy consumption by
orders of magnitude with respect to general-purpose processors and GPUs, and it
requires a very small chip area. Compared to a modern mobile SoC, an NVIDIA
Tegra X1, E-PUR provides an average energy reduction of 92x.
On the State of the Art of Evaluation in Neural Language Models
Authors: Gábor Melis, Chris Dyer, Phil Blunsom
Date: 2017-07-18 00:00:00
Proceeding: ICLR 2018 1
Tasks: Language Modelling

Ongoing innovations in recurrent neural network architectures have provided a
steady influx of apparently state-of-the-art results on language modelling
benchmarks. However, these have been evaluated using differing code bases and
limited computational resources, which represent uncontrolled sources of
experimental variation. We reevaluate several popular architectures and
regularisation methods with large-scale automatic black-box hyperparameter
tuning and arrive at the somewhat surprising conclusion that standard LSTM
architectures, when properly regularised, outperform more recent models. We
establish a new state of the art on the Penn Treebank and Wikitext-2 corpora,
as well as strong baselines on the Hutter Prize dataset.
On Convergence of Epanechnikov Mean Shift
Authors: Kejun Huang, Xiao Fu, Nicholas D. Sidiropoulos
Date: 2017-11-20 00:00:00
Tasks: Clustering

Epanechnikov Mean Shift is a simple yet empirically very effective algorithm
for clustering. It localizes the centroids of data clusters via estimating
modes of the probability distribution that generates the data points, using the
`optimal' Epanechnikov kernel density estimator. However, since the procedure
involves non-smooth kernel density functions, the convergence behavior of
Epanechnikov mean shift lacks theoretical support as of this writing---most of
the existing analyses are based on smooth functions and thus cannot be applied
to Epanechnikov Mean Shift. In this work, we first show that the original
Epanechnikov Mean Shift may indeed terminate at a non-critical point, due to
the non-smoothness nature. Based on our analysis, we propose a simple remedy to
fix it. The modified Epanechnikov Mean Shift is guaranteed to terminate at a
local maximum of the estimated density, which corresponds to a cluster
centroid, within a finite number of iterations. We also propose a way to avoid
running the Mean Shift iterates from every data point, while maintaining good
clustering accuracies under non-overlapping spherical Gaussian mixture models.
This further pushes Epanechnikov Mean Shift to handle very large and
high-dimensional data sets. Experiments show surprisingly good performance
compared to the Lloyd's K-means algorithm and the EM algorithm.
Deep Reinforcement Learning for Multi-Resource Multi-Machine Job Scheduling
Authors: Weijia Chen, Yuedong Xu, Xiaofeng Wu
Date: 2017-11-20 00:00:00
Tasks: CPU, Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Scheduling

Minimizing job scheduling time is a fundamental issue in data center networks
that has been extensively studied in recent years. The incoming jobs require
different CPU and memory units, and span different number of time slots. The
traditional solution is to design efficient heuristic algorithms with
performance guarantee under certain assumptions. In this paper, we improve a
recently proposed job scheduling algorithm using deep reinforcement learning
and extend it to multiple server clusters. Our study reveals that deep
reinforcement learning method has the potential to outperform traditional
resource allocation algorithms in a variety of complicated environments.
Deep Approximately Orthogonal Nonnegative Matrix Factorization for Clustering
Authors: Yuning Qiu, Guoxu Zhou, Kan Xie
Date: 2017-11-20 00:00:00
Tasks: Clustering

Nonnegative Matrix Factorization (NMF) is a widely used technique for data
representation. Inspired by the expressive power of deep learning, several NMF
variants equipped with deep architectures have been proposed. However, these
methods mostly use the only nonnegativity while ignoring task-specific features
of data. In this paper, we propose a novel deep approximately orthogonal
nonnegative matrix factorization method where both nonnegativity and
orthogonality are imposed with the aim to perform a hierarchical clustering by
using different level of abstractions of data. Experiment on two face image
datasets showed that the proposed method achieved better clustering performance
than other deep matrix factorization methods and state-of-the-art single layer
NMF variants.
Relaxed Oracles for Semi-Supervised Clustering
Authors: Taewan Kim, Joydeep Ghosh
Date: 2017-11-20 00:00:00
Tasks: Clustering

Pairwise "same-cluster" queries are one of the most widely used forms of
supervision in semi-supervised clustering. However, it is impractical to ask
human oracles to answer every query correctly. In this paper, we study the
influence of allowing "not-sure" answers from a weak oracle and propose an
effective algorithm to handle such uncertainties in query responses. Two
realistic weak oracle models are considered where ambiguity in answering
depends on the distance between two points. We show that a small query
complexity is adequate for effective clustering with high probability by
providing better pairs to the weak oracle. Experimental results on synthetic
and real data show the effectiveness of our approach in overcoming supervision
uncertainties and yielding high quality clusters.
Action Recognition with Coarse-to-Fine Deep Feature Integration and Asynchronous Fusion
Authors: Weiyao Lin, Yang Mi, Jianxin Wu, Ke Lu, Hongkai Xiong
Date: 2017-11-20 00:00:00
Tasks: Action Recognition, Temporal Action Localization

Action recognition is an important yet challenging task in computer vision.
In this paper, we propose a novel deep-based framework for action recognition,
which improves the recognition accuracy by: 1) deriving more precise features
for representing actions, and 2) reducing the asynchrony between different
information streams. We first introduce a coarse-to-fine network which extracts
shared deep features at different action class granularities and progressively
integrates them to obtain a more accurate feature representation for input
actions. We further introduce an asynchronous fusion network. It fuses
information from different streams by asynchronously integrating stream-wise
features at different time points, hence better leveraging the complementary
information in different streams. Experimental results on action recognition
benchmarks demonstrate that our approach achieves the state-of-the-art
performance.
ConvAMR: Abstract meaning representation parsing for legal document
Authors: Lai Dac Viet, Vu Trong Sinh, Nguyen Le Minh, Ken Satoh
Date: 2017-11-16 00:00:00
Tasks: Abstract Meaning Representation

Convolutional neural networks (CNN) have recently achieved remarkable
performance in a wide range of applications. In this research, we equip
convolutional sequence-to-sequence (seq2seq) model with an efficient graph
linearization technique for abstract meaning representation parsing. Our
linearization method is better than the prior method at signaling the turn of
graph traveling. Additionally, convolutional seq2seq model is more appropriate
and considerably faster than the recurrent neural network models in this task.
Our method outperforms previous methods by a large margin on both the standard
dataset LDC2014T12. Our result indicates that future works still have a room
for improving parsing model using graph linearization approach.
Liver lesion segmentation informed by joint liver segmentation
Authors: Eugene Vorontsov, An Tang, Chris Pal, Samuel Kadoury
Date: 2017-07-24 00:00:00
Tasks: Computed Tomography (CT), Lesion Detection, Lesion Segmentation, Liver Segmentation, Segmentation

We propose a model for the joint segmentation of the liver and liver lesions
in computed tomography (CT) volumes. We build the model from two fully
convolutional networks, connected in tandem and trained together end-to-end. We
evaluate our approach on the 2017 MICCAI Liver Tumour Segmentation Challenge,
attaining competitive liver and liver lesion detection and segmentation scores
across a wide range of metrics. Unlike other top performing methods, our model
output post-processing is trivial, we do not use data external to the
challenge, and we propose a simple single-stage model that is trained
end-to-end. However, our method nearly matches the top lesion segmentation
performance and achieves the second highest precision for lesion detection
while maintaining high recall.
Robust Seed Mask Generation for Interactive Image Segmentation
Authors: Mario Amrehn, Stefan Steidl, Markus Kowarschik, Andreas Maier
Date: 2017-11-20 00:00:00
Tasks: Image Segmentation, Medical Image Segmentation, Object, Segmentation, Semantic Segmentation

In interactive medical image segmentation, anatomical structures are
extracted from reconstructed volumetric images. The first iterations of user
interaction traditionally consist of drawing pictorial hints as an initial
estimate of the object to extract. Only after this time consuming first phase,
the efficient selective refinement of current segmentation results begins.
Erroneously labeled seeds, especially near the border of the object, are
challenging to detect and replace for a human and may substantially impact the
overall segmentation quality. We propose an automatic seeding pipeline as well
as a configuration based on saliency recognition, in order to skip the
time-consuming initial interaction phase during segmentation. A median Dice
score of 68.22% is reached before the first user interaction on the test data
set with an error rate in seeding of only 0.088%.