pwc-archive/papers-with-abstracts huggingface dataset:
Incorporating Network Built-in Priors in Weakly-supervised Semantic Segmentation
Authors: Fatemeh Sadat Saleh, Mohammad Sadegh Aliakbarian, Mathieu Salzmann, Lars Petersson, Jose M. Alvarez, Stephen Gould
Date: 2017-06-06 00:00:00
Tasks: Object Recognition, Segmentation, Semantic Segmentation, TAG, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Pixel-level annotations are expensive and time consuming to obtain. Hence,
weak supervision using only image tags could have a significant impact in
semantic segmentation. Recently, CNN-based methods have proposed to fine-tune
pre-trained networks using image tags. Without additional information, this
leads to poor localization accuracy. This problem, however, was alleviated by
making use of objectness priors to generate foreground/background masks.
Unfortunately these priors either require pixel-level annotations/bounding
boxes, or still yield inaccurate object boundaries. Here, we propose a novel
method to extract accurate masks from networks pre-trained for the task of
object recognition, thus forgoing external objectness modules. We first show
how foreground/background masks can be obtained from the activations of
higher-level convolutional layers of a network. We then show how to obtain
multi-class masks by the fusion of foreground/background ones with information
extracted from a weakly-supervised localization network. Our experiments
evidence that exploiting these masks in conjunction with a weakly-supervised
training loss yields state-of-the-art tag-based weakly-supervised semantic
segmentation results.
Corralling a Band of Bandit Algorithms
Authors: Alekh Agarwal, Haipeng Luo, Behnam Neyshabur, Robert E. Schapire
Date: 2016-12-19 00:00:00
Tasks: Multi-Armed Bandits

We study the problem of combining multiple bandit algorithms (that is, online
learning algorithms with partial feedback) with the goal of creating a master
algorithm that performs almost as well as the best base algorithm if it were to
be run on its own. The main challenge is that when run with a master, base
algorithms unavoidably receive much less feedback and it is thus critical that
the master not starve a base algorithm that might perform uncompetitively
initially but would eventually outperform others if given enough feedback. We
address this difficulty by devising a version of Online Mirror Descent with a
special mirror map together with a sophisticated learning rate scheme. We show
that this approach manages to achieve a more delicate balance between
exploiting and exploring base algorithms than previous works yielding superior
regret bounds.
  Our results are applicable to many settings, such as multi-armed bandits,
contextual bandits, and convex bandits. As examples, we present two main
applications. The first is to create an algorithm that enjoys worst-case
robustness while at the same time performing much better when the environment
is relatively easy. The second is to create an algorithm that works
simultaneously under different assumptions of the environment, such as
different priors or different loss structures.
Attention-over-Attention Neural Networks for Reading Comprehension
Authors: Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, Guoping Hu
Date: 2016-07-15 00:00:00
Proceeding: ACL 2017 7
Tasks: Question Answering, Reading Comprehension

Cloze-style queries are representative problems in reading comprehension.
Over the past few months, we have seen much progress that utilizing neural
network approach to solve Cloze-style questions. In this paper, we present a
novel model called attention-over-attention reader for the Cloze-style reading
comprehension task. Our model aims to place another attention mechanism over
the document-level attention, and induces "attended attention" for final
predictions. Unlike the previous works, our neural network model requires less
pre-defined hyper-parameters and uses an elegant architecture for modeling.
Experimental results show that the proposed attention-over-attention model
significantly outperforms various state-of-the-art systems by a large margin in
public datasets, such as CNN and Children's Book Test datasets.
Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution
Authors: Ting Liu, Yiming Cui, Qingyu Yin, Wei-Nan Zhang, Shijin Wang, Guoping Hu
Date: 2016-06-06 00:00:00
Proceeding: ACL 2017 7
Tasks: Reading Comprehension

Most existing approaches for zero pronoun resolution are heavily relying on
annotated data, which is often released by shared task organizers. Therefore,
the lack of annotated data becomes a major obstacle in the progress of zero
pronoun resolution task. Also, it is expensive to spend manpower on labeling
the data for better performance. To alleviate the problem above, in this paper,
we propose a simple but novel approach to automatically generate large-scale
pseudo training data for zero pronoun resolution. Furthermore, we successfully
transfer the cloze-style reading comprehension neural network model into zero
pronoun resolution task and propose a two-step training mechanism to overcome
the gap between the pseudo training data and the real one. Experimental results
show that the proposed approach significantly outperforms the state-of-the-art
systems with an absolute improvements of 3.1% F-score on OntoNotes 5.0 data.
Classifying Documents within Multiple Hierarchical Datasets using Multi-Task Learning
Authors: Azad Naik, Anveshi Charuvaka, Huzefa Rangwala
Date: 2017-06-06 00:00:00
Tasks: Binary Classification, Classification, General Classification, Multi-class Classification, Multi-Task Learning, Transfer Learning

Multi-task learning (MTL) is a supervised learning paradigm in which the
prediction models for several related tasks are learned jointly to achieve
better generalization performance. When there are only a few training examples
per task, MTL considerably outperforms the traditional Single task learning
(STL) in terms of prediction accuracy. In this work we develop an MTL based
approach for classifying documents that are archived within dual concept
hierarchies, namely, DMOZ and Wikipedia. We solve the multi-class
classification problem by defining one-versus-rest binary classification tasks
for each of the different classes across the two hierarchical datasets. Instead
of learning a linear discriminant for each of the different tasks
independently, we use a MTL approach with relationships between the different
tasks across the datasets established using the non-parametric, lazy, nearest
neighbor approach. We also develop and evaluate a transfer learning (TL)
approach and compare the MTL (and TL) methods against the standard single task
learning and semi-supervised learning approaches. Our empirical results
demonstrate the strength of our developed methods that show an improvement
especially when there are fewer number of training examples per classification
task.
Embedding Feature Selection for Large-scale Hierarchical Classification
Authors: Azad Naik, Huzefa Rangwala
Date: 2017-06-06 00:00:00
Tasks: Classification, Dimensionality Reduction, feature selection, General Classification

Large-scale Hierarchical Classification (HC) involves datasets consisting of
thousands of classes and millions of training instances with high-dimensional
features posing several big data challenges. Feature selection that aims to
select the subset of discriminant features is an effective strategy to deal
with large-scale HC problem. It speeds up the training process, reduces the
prediction time and minimizes the memory requirements by compressing the total
size of learned model weight vectors. Majority of the studies have also shown
feature selection to be competent and successful in improving the
classification accuracy by removing irrelevant features. In this work, we
investigate various filter-based feature selection methods for dimensionality
reduction to solve the large-scale HC problem. Our experimental evaluation on
text and image datasets with varying distribution of features, classes and
instances shows upto 3x order of speed-up on massive datasets and upto 45% less
memory requirements for storing the weight vectors of learned model without any
significant loss (improvement for some datasets) in the classification
accuracy. Source Code: https://cs.gmu.edu/~mlbio/featureselection.
Online Learning Without Prior Information
Authors: Ashok Cutkosky, Kwabena Boahen
Date: 2017-03-07 00:00:00

The vast majority of optimization and online learning algorithms today
require some prior information about the data (often in the form of bounds on
gradients or on the optimal parameter value). When this information is not
available, these algorithms require laborious manual tuning of various
hyperparameters, motivating the search for algorithms that can adapt to the
data with no prior information. We describe a frontier of new lower bounds on
the performance of such algorithms, reflecting a tradeoff between a term that
depends on the optimal parameter value and a term that depends on the
gradients' rate of growth. Further, we construct a family of algorithms whose
performance matches any desired point on this frontier, which no previous
algorithm reaches.
Geometric Multi-Model Fitting with a Convex Relaxation Algorithm
Authors: Paul Amayo, Pedro Pinies, Lina M. Paz, Paul Newman
Date: 2017-06-05 00:00:00
Proceeding: CVPR 2018 6
Tasks: Homography Estimation

We propose a novel method to fit and segment multi-structural data via convex
relaxation. Unlike greedy methods --which maximise the number of inliers-- this
approach efficiently searches for a soft assignment of points to models by
minimising the energy of the overall classification. Our approach is similar to
state-of-the-art energy minimisation techniques which use a global energy.
However, we deal with the scaling factor (as the number of models increases) of
the original combinatorial problem by relaxing the solution. This relaxation
brings two advantages: first, by operating in the continuous domain we can
parallelize the calculations. Second, it allows for the use of different
metrics which results in a more general formulation.
  We demonstrate the versatility of our technique on two different problems of
estimating structure from images: plane extraction from RGB-D data and
homography estimation from pairs of images. In both cases, we report accurate
results on publicly available datasets, in most of the cases outperforming the
state-of-the-art.
Distributed Graph Clustering by Load Balancing
Authors: He Sun, Luca Zanetti
Date: 2016-07-18 00:00:00
Tasks: Clustering, Distributed Computing, Graph Clustering

Graph clustering is a fundamental computational problem with a number of
applications in algorithm design, machine learning, data mining, and analysis
of social networks. Over the past decades, researchers have proposed a number
of algorithmic design methods for graph clustering. However, most of these
methods are based on complicated spectral techniques or convex optimisation,
and cannot be applied directly for clustering many networks that occur in
practice, whose information is often collected on different sites. Designing a
simple and distributed clustering algorithm is of great interest, and has wide
applications for processing big datasets. In this paper we present a simple and
distributed algorithm for graph clustering: for a wide class of graphs that are
characterised by a strong cluster-structure, our algorithm finishes in a
poly-logarithmic number of rounds, and recovers a partition of the graph close
to an optimal partition. The main component of our algorithm is an application
of the random matching model of load balancing, which is a fundamental protocol
in distributed computing and has been extensively studied in the past 20 years.
Hence, our result highlights an intrinsic and interesting connection between
graph clustering and load balancing. At a technical level, we present a purely
algebraic result characterising the early behaviours of load balancing
processes for graphs exhibiting a cluster-structure. We believe that this
result can be further applied to analyse other gossip processes, such as rumour
spreading and averaging processes.
Progressive Boosting for Class Imbalance
Authors: Roghayeh Soleymani, Eric Granger, Giorgio Fumera
Date: 2017-06-05 00:00:00
Tasks: Ensemble Learning

Pattern recognition applications often suffer from skewed data distributions
between classes, which may vary during operations w.r.t. the design data.
Two-class classification systems designed using skewed data tend to recognize
the majority class better than the minority class of interest. Several
data-level techniques have been proposed to alleviate this issue by up-sampling
minority samples or under-sampling majority samples. However, some informative
samples may be neglected by random under-sampling and adding synthetic positive
samples through up-sampling adds to training complexity. In this paper, a new
ensemble learning algorithm called Progressive Boosting (PBoost) is proposed
that progressively inserts uncorrelated groups of samples into a Boosting
procedure to avoid loss of information while generating a diverse pool of
classifiers. Base classifiers in this ensemble are generated from one iteration
to the next, using subsets from a validation set that grows gradually in size
and imbalance. Consequently, PBoost is more robust to unknown and variable
levels of skew in operational data, and has lower computation complexity than
Boosting ensembles in literature. In PBoost, a new loss factor is proposed to
avoid bias of performance towards the negative class. Using this loss factor,
the weight update of samples and classifier contribution in final predictions
are set based on the ability to recognize both classes. Using the proposed loss
factor instead of standard accuracy can avoid biasing performance in any
Boosting ensemble. The proposed approach was validated and compared using
synthetic data, videos from the FIA dataset that emulates face
re-identification applications, and KEEL collection of datasets. Results show
that PBoost can outperform state of the art techniques in terms of both
accuracy and complexity over different levels of imbalance and overlap between
classes.
End-to-end learning of brain tissue segmentation from imperfect labeling
Authors: Alex Fedorov, Jeremy Johnson, Eswar Damaraju, Alexei Ozerin, Vince Calhoun, Sergey Plis
Date: 2016-12-03 00:00:00

Segmenting a structural magnetic resonance imaging (MRI) scan is an important
pre-processing step for analytic procedures and subsequent inferences about
longitudinal tissue changes. Manual segmentation defines the current gold
standard in quality but is prohibitively expensive. Automatic approaches are
computationally intensive, incredibly slow at scale, and error prone due to
usually involving many potentially faulty intermediate steps. In order to
streamline the segmentation, we introduce a deep learning model that is based
on volumetric dilated convolutions, subsequently reducing both processing time
and errors. Compared to its competitors, the model has a reduced set of
parameters and thus is easier to train and much faster to execute. The contrast
in performance between the dilated network and its competitors becomes obvious
when both are tested on a large dataset of unprocessed human brain volumes. The
dilated network consistently outperforms not only another state-of-the-art deep
learning approach, the up convolutional network, but also the ground truth on
which it was trained. Not only can the incredible speed of our model make large
scale analyses much easier but we also believe it has great potential in a
clinical setting where, with little to no substantial delay, a patient and
provider can go over test results.
Learning Multivariate Log-concave Distributions
Authors: Ilias Diakonikolas, Daniel M. Kane, Alistair Stewart
Date: 2016-05-26 00:00:00

We study the problem of estimating multivariate log-concave probability
density functions. We prove the first sample complexity upper bound for
learning log-concave densities on $\mathbb{R}^d$, for all $d \geq 1$. Prior to
our work, no upper bound on the sample complexity of this learning problem was
known for the case of $d>3$. In more detail, we give an estimator that, for any
$d \ge 1$ and $\epsilon>0$, draws $\tilde{O}_d \left( (1/\epsilon)^{(d+5)/2}
\right)$ samples from an unknown target log-concave density on $\mathbb{R}^d$,
and outputs a hypothesis that (with high probability) is $\epsilon$-close to
the target, in total variation distance. Our upper bound on the sample
complexity comes close to the known lower bound of $\Omega_d \left(
(1/\epsilon)^{(d+1)/2} \right)$ for this problem.
Facial Emotion Detection Using Convolutional Neural Networks and Representational Autoencoder Units
Authors: Prudhvi Raj Dachapally
Date: 2017-06-05 00:00:00
Tasks: Emotion Recognition

Emotion being a subjective thing, leveraging knowledge and science behind
labeled data and extracting the components that constitute it, has been a
challenging problem in the industry for many years. With the evolution of deep
learning in computer vision, emotion recognition has become a widely-tackled
research problem. In this work, we propose two independent methods for this
very task. The first method uses autoencoders to construct a unique
representation of each emotion, while the second method is an 8-layer
convolutional neural network (CNN). These methods were trained on the
posed-emotion dataset (JAFFE), and to test their robustness, both the models
were also tested on 100 random images from the Labeled Faces in the Wild (LFW)
dataset, which consists of images that are candid than posed. The results show
that with more fine-tuning and depth, our CNN model can outperform the
state-of-the-art methods for emotion recognition. We also propose some exciting
ideas for expanding the concept of representational autoencoders to improve
their performance.
Residual LSTM: Design of a Deep Recurrent Architecture for Distant Speech Recognition
Authors: Jaeyoung Kim, Mostafa El-Khamy, Jungwon Lee
Date: 2017-01-10 00:00:00
Tasks: Distant Speech Recognition, speech-recognition, Speech Recognition

In this paper, a novel architecture for a deep recurrent neural network,
residual LSTM is introduced. A plain LSTM has an internal memory cell that can
learn long term dependencies of sequential data. It also provides a temporal
shortcut path to avoid vanishing or exploding gradients in the temporal domain.
The residual LSTM provides an additional spatial shortcut path from lower
layers for efficient training of deep networks with multiple LSTM layers.
Compared with the previous work, highway LSTM, residual LSTM separates a
spatial shortcut path with temporal one by using output layers, which can help
to avoid a conflict between spatial and temporal-domain gradient flows.
Furthermore, residual LSTM reuses the output projection matrix and the output
gate of LSTM to control the spatial information flow instead of additional gate
networks, which effectively reduces more than 10% of network parameters. An
experiment for distant speech recognition on the AMI SDM corpus shows that
10-layer plain and highway LSTM networks presented 13.7% and 6.2% increase in
WER over 3-layer aselines, respectively. On the contrary, 10-layer residual
LSTM networks provided the lowest WER 41.0%, which corresponds to 3.3% and 2.8%
WER reduction over plain and highway LSTM networks, respectively.
Read, Tag, and Parse All at Once, or Fully-neural Dependency Parsing
Authors: Jan Chorowski, Michał Zapotoczny, Paweł Rychlikowski
Date: 2016-09-12 00:00:00
Tasks: All, Dependency Parsing, Part-Of-Speech Tagging, POS, POS Tagging, TAG

We present a dependency parser implemented as a single deep neural network
that reads orthographic representations of words and directly generates
dependencies and their labels. Unlike typical approaches to parsing, the model
doesn't require part-of-speech (POS) tagging of the sentences. With proper
regularization and additional supervision achieved with multitask learning we
reach state-of-the-art performance on Slavic languages from the Universal
Dependencies treebank: with no linguistic features other than characters, our
parser is as accurate as a transition- based system trained on perfect POS
tags.
Using CMA-ES for tuning coupled PID controllers within models of combustion engines
Authors: Katerina Henclova
Date: 2016-09-21 00:00:00

Proportional integral derivative (PID) controllers are important and widely
used tools in system control. Tuning of the controller gains is a laborious
task, especially for complex systems such as combustion engines. To minimize
the time of an engineer for tuning of the gains in a simulation software, we
propose to formulate a part of the problem as a black-box optimization task. In
this paper, we summarize the properties and practical limitations of tuning of
the gains in this particular application. We investigate the latest methods of
black-box optimization and conclude that the Covariance Matrix Adaptation
Evolution Strategy (CMA-ES) with bi-population restart strategy, elitist parent
selection and active covariance matrix adaptation is best suited for this task.
Details of the algorithm's experiment-based calibration are explained as well
as derivation of a suitable objective function. The method's performance is
compared with that of PSO and SHADE. Finally, its usability is verified on six
models of real engines.
PixColor: Pixel Recursive Colorization
Authors: Sergio Guadarrama, Ryan Dahl, David Bieber, Mohammad Norouzi, Jonathon Shlens, Kevin Murphy
Date: 2017-05-19 00:00:00
Tasks: Colorization

We propose a novel approach to automatically produce multiple colorized
versions of a grayscale image. Our method results from the observation that the
task of automated colorization is relatively easy given a low-resolution
version of the color image. We first train a conditional PixelCNN to generate a
low resolution color for a given grayscale image. Then, given the generated
low-resolution color image and the original grayscale image as inputs, we train
a second CNN to generate a high-resolution colorization of an image. We
demonstrate that our approach produces more diverse and plausible colorizations
than existing methods, as judged by human raters in a "Visual Turing Test".
Visual attention models for scene text recognition
Authors: Suman K. Ghosh, Ernest Valveny, Andrew D. Bagdanov
Date: 2017-06-05 00:00:00
Tasks: Language Modeling, Language Modelling, Scene Text Recognition

In this paper we propose an approach to lexicon-free recognition of text in
scene images. Our approach relies on a LSTM-based soft visual attention model
learned from convolutional features. A set of feature vectors are derived from
an intermediate convolutional layer corresponding to different areas of the
image. This permits encoding of spatial information into the image
representation. In this way, the framework is able to learn how to selectively
focus on different parts of the image. At every time step the recognizer emits
one character using a weighted combination of the convolutional feature vectors
according to the learned attention model. Training can be done end-to-end using
only word level annotations. In addition, we show that modifying the beam
search algorithm by integrating an explicit language model leads to
significantly better recognition results. We validate the performance of our
approach on standard SVT and ICDAR'03 scene text datasets, showing
state-of-the-art performance in unconstrained text recognition.