pwc-archive/papers-with-abstracts huggingface dataset:
Using Filter Banks in Convolutional Neural Networks for Texture Classification
Authors: Vincent Andrearczyk, Paul F. Whelan
Date: 2016-01-12 00:00:00
Tasks: Classification, General Classification, Object, object-detection, Object Detection, speech-recognition, Speech Recognition, Texture Classification

Deep learning has established many new state of the art solutions in the last
decade in areas such as object, scene and speech recognition. In particular
Convolutional Neural Network (CNN) is a category of deep learning which obtains
excellent results in object detection and recognition tasks. Its architecture
is indeed well suited to object analysis by learning and classifying complex
(deep) features that represent parts of an object or the object itself.
However, some of its features are very similar to texture analysis methods. CNN
layers can be thought of as filter banks of complexity increasing with the
depth. Filter banks are powerful tools to extract texture features and have
been widely used in texture analysis. In this paper we develop a simple network
architecture named Texture CNN (T-CNN) which explores this observation. It is
built on the idea that the overall shape information extracted by the fully
connected layers of a classic CNN is of minor importance in texture analysis.
Therefore, we pool an energy measure from the last convolution layer which we
connect to a fully connected layer. We show that our approach can improve the
performance of a network while greatly reducing the memory usage and
computation.
Constraint-Based Clustering Selection
Authors: Toon Van Craenendonck, Hendrik Blockeel
Date: 2016-09-23 00:00:00
Tasks: Clustering

Semi-supervised clustering methods incorporate a limited amount of
supervision into the clustering process. Typically, this supervision is
provided by the user in the form of pairwise constraints. Existing methods use
such constraints in one of the following ways: they adapt their clustering
procedure, their similarity metric, or both. All of these approaches operate
within the scope of individual clustering algorithms. In contrast, we propose
to use constraints to choose between clusterings generated by very different
unsupervised clustering algorithms, run with different parameter settings. We
empirically show that this simple approach often outperforms existing
semi-supervised clustering methods.
On the (im)possibility of fairness
Authors: Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian
Date: 2016-09-23 00:00:00
Tasks: Decision Making, Fairness

What does it mean for an algorithm to be fair? Different papers use different
notions of algorithmic fairness, and although these appear internally
consistent, they also seem mutually incompatible. We present a mathematical
setting in which the distinctions in previous papers can be made formal. In
addition to characterizing the spaces of inputs (the "observed" space) and
outputs (the "decision" space), we introduce the notion of a construct space: a
space that captures unobservable, but meaningful variables for the prediction.
  We show that in order to prove desirable properties of the entire
decision-making process, different mechanisms for fairness require different
assumptions about the nature of the mapping from construct space to decision
space. The results in this paper imply that future treatments of algorithmic
fairness should more explicitly state assumptions about the relationship
between constructs and observations.
Deep Multi-Task Learning with Shared Memory
Authors: Pengfei Liu, Xipeng Qiu, Xuanjing Huang
Date: 2016-09-23 00:00:00
Tasks: General Classification, Multi-Task Learning, text-classification, Text Classification

Neural network based models have achieved impressive results on various
specific tasks. However, in previous works, most models are learned separately
based on single-task supervised objectives, which often suffer from
insufficient training data. In this paper, we propose two deep architectures
which can be trained jointly on multiple related tasks. More specifically, we
augment neural model with an external memory, which is shared by several tasks.
Experiments on two groups of text classification tasks show that our proposed
architectures can improve the performance of a task with the help of other
related tasks.
A Novel Progressive Multi-label Classifier for Classincremental Data
Authors: Mihika Dave, Sahil Tapiawala, Meng Joo Er, Rajasekar Venkatesan
Date: 2016-09-23 00:00:00
Tasks: class-incremental learning, Class Incremental Learning, General Classification, Incremental Learning, Multi-Label Classification, MUlTI-LABEL-ClASSIFICATION

In this paper, a progressive learning algorithm for multi-label
classification to learn new labels while retaining the knowledge of previous
labels is designed. New output neurons corresponding to new labels are added
and the neural network connections and parameters are automatically
restructured as if the label has been introduced from the beginning. This work
is the first of the kind in multi-label classifier for class-incremental
learning. It is useful for real-world applications such as robotics where
streaming data are available and the number of labels is often unknown. Based
on the Extreme Learning Machine framework, a novel universal classifier with
plug and play capabilities for progressive multi-label classification is
developed. Experimental results on various benchmark synthetic and real
datasets validate the efficiency and effectiveness of our proposed algorithm.
Understanding Symmetric Smoothing Filters: A Gaussian Mixture Model Perspective
Authors: Stanley H. Chan, Todd Zickler, Yue M. Lu
Date: 2016-01-01 00:00:00
Tasks: Denoising, Image Denoising, Unity

Many patch-based image denoising algorithms can be formulated as applying a
smoothing filter to the noisy image. Expressed as matrices, the smoothing
filters must be row normalized so that each row sums to unity. Surprisingly, if
we apply a column normalization before the row normalization, the performance
of the smoothing filter can often be significantly improved. Prior works showed
that such performance gain is related to the Sinkhorn-Knopp balancing
algorithm, an iterative procedure that symmetrizes a row-stochastic matrix to a
doubly-stochastic matrix. However, a complete understanding of the performance
gain phenomenon is still lacking.
  In this paper, we study the performance gain phenomenon from a statistical
learning perspective. We show that Sinkhorn-Knopp is equivalent to an
Expectation-Maximization (EM) algorithm of learning a Gaussian mixture model of
the image patches. By establishing the correspondence between the steps of
Sinkhorn-Knopp and the EM algorithm, we provide a geometrical interpretation of
the symmetrization process. This observation allows us to develop a new
denoising algorithm called Gaussian mixture model symmetric smoothing filter
(GSF). GSF is an extension of the Sinkhorn-Knopp and is a generalization of the
original smoothing filters. Despite its simple formulation, GSF outperforms
many existing smoothing filters and has a similar performance compared to
several state-of-the-art denoising algorithms.
Multilayer Spectral Graph Clustering via Convex Layer Aggregation
Authors: Pin-Yu Chen, Alfred O. Hero III
Date: 2016-09-23 00:00:00
Tasks: Clustering, Graph Clustering, Spectral Graph Clustering

Multilayer graphs are commonly used for representing different relations
between entities and handling heterogeneous data processing tasks. New
challenges arise in multilayer graph clustering for assigning clusters to a
common multilayer node set and for combining information from each layer. This
paper presents a theoretical framework for multilayer spectral graph clustering
of the nodes via convex layer aggregation. Under a novel multilayer signal plus
noise model, we provide a phase transition analysis that establishes the
existence of a critical value on the noise level that permits reliable cluster
separation. The analysis also specifies analytical upper and lower bounds on
the critical value, where the bounds become exact when the clusters have
identical sizes. Numerical experiments on synthetic multilayer graphs are
conducted to validate the phase transition analysis and study the effect of
layer weights and noise levels on clustering reliability.
Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regression
Authors: Mahsa Taheri, Néhémy Lim, Johannes Lederer
Date: 2016-09-23 00:00:00
Tasks: Astronomy, feature selection, regression

Modern technologies are generating ever-increasing amounts of data. Making use of these data requires methods that are both statistically sound and computationally efficient. Typically, the statistical and computational aspects are treated separately. In this paper, we propose an approach to entangle these two aspects in the context of regularized estimation. Applying our approach to sparse and group-sparse regression, we show that it can improve on standard pipelines both statistically and computationally.
Data-dependent Initializations of Convolutional Neural Networks
Authors: Philipp Krähenbühl, Carl Doersch, Jeff Donahue, Trevor Darrell
Date: 2015-11-21 00:00:00
Tasks: image-classification, Image Classification, object-detection, Object Detection, Unsupervised Pre-training

Convolutional Neural Networks spread through computer vision like a wildfire,
impacting almost all visual tasks imaginable. Despite this, few researchers
dare to train their models from scratch. Most work builds on one of a handful
of ImageNet pre-trained models, and fine-tunes or adapts these for specific
tasks. This is in large part due to the difficulty of properly initializing
these networks from scratch. A small miscalibration of the initial weights
leads to vanishing or exploding gradients, as well as poor convergence
properties. In this work we present a fast and simple data-dependent
initialization procedure, that sets the weights of a network such that all
units in the network train at roughly the same rate, avoiding vanishing or
exploding gradients. Our initialization matches the current state-of-the-art
unsupervised or self-supervised pre-training methods on standard computer
vision tasks, such as image classification and object detection, while being
roughly three orders of magnitude faster. When combined with pre-training
methods, our initialization significantly outperforms prior work, narrowing the
gap between supervised and unsupervised pre-training.
Deep Quality: A Deep No-reference Quality Assessment System
Authors: Prajna Paramita Dash, Akshaya Mishra, Alexander Wong
Date: 2016-09-22 00:00:00
Tasks: Image Quality Assessment, No-Reference Image Quality Assessment

Image quality assessment (IQA) continues to garner great interest in the
research community, particularly given the tremendous rise in consumer video
capture and streaming. Despite significant research effort in IQA in the past
few decades, the area of no-reference image quality assessment remains a great
challenge and is largely unsolved. In this paper, we propose a novel
no-reference image quality assessment system called Deep Quality, which
leverages the power of deep learning to model the complex relationship between
visual content and the perceived quality. Deep Quality consists of a novel
multi-scale deep convolutional neural network, trained to learn to assess image
quality based on training samples consisting of different distortions and
degradations such as blur, Gaussian noise, and compression artifacts.
Preliminary results using the CSIQ benchmark image quality dataset showed that
Deep Quality was able to achieve strong quality prediction performance (89%
patch-level and 98% image-level prediction accuracy), being able to achieve
similar performance as full-reference IQA methods.
Robust Confidence Intervals in High-Dimensional Left-Censored Regression
Authors: Jelena Bradic, Jiaqi Guo
Date: 2016-09-22 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

This paper develops robust confidence intervals in high-dimensional and
left-censored regression. Type-I censored regression models are extremely
common in practice, where a competing event makes the variable of interest
unobservable. However, techniques developed for entirely observed data do not
directly apply to the censored observations. In this paper, we develop smoothed
estimating equations that augment the de-biasing method, such that the
resulting estimator is adaptive to censoring and is more robust to the
misspecification of the error distribution. We propose a unified class of
robust estimators, including Mallow's, Schweppe's and Hill-Ryan's one-step
estimator.
  In the ultra-high-dimensional setting, where the dimensionality can grow
exponentially with the sample size, we show that as long as the preliminary
estimator converges faster than $n^{-1/4}$, the one-step estimator inherits
asymptotic distribution of fully iterated version. Moreover, we show that the
size of the residuals of the Bahadur representation matches those of the simple
linear models, $s^{3/4 } (\log (p \vee n))^{3/4} / n^{1/4}$ -- that is, the
effects of censoring asymptotically disappear. Simulation studies demonstrate
that our method is adaptive to the censoring level and asymmetry in the error
distribution, and does not lose efficiency when the errors are from symmetric
distributions. Finally, we apply the developed method to a real data set from
the MAQC-II repository that is related to the HIV-1 study.
The Importance of Skip Connections in Biomedical Image Segmentation
Authors: Michal Drozdzal, Eugene Vorontsov, Gabriel Chartrand, Samuel Kadoury, Chris Pal
Date: 2016-08-14 00:00:00
Tasks: Image Segmentation, Semantic Segmentation

In this paper, we study the influence of both long and short skip connections
on Fully Convolutional Networks (FCN) for biomedical image segmentation. In
standard FCNs, only long skip connections are used to skip features from the
contracting path to the expanding path in order to recover spatial information
lost during downsampling. We extend FCNs by adding short skip connections, that
are similar to the ones introduced in residual networks, in order to build very
deep FCNs (of hundreds of layers). A review of the gradient flow confirms that
for a very deep FCN it is beneficial to have both long and short skip
connections. Finally, we show that a very deep FCN can achieve
near-to-state-of-the-art results on the EM dataset without any further
post-processing.
A Fully Convolutional Neural Network for Speech Enhancement
Authors: Se Rim Park, Jinwon Lee
Date: 2016-09-22 00:00:00
Tasks: Decoder, Speech Enhancement

In hearing aids, the presence of babble noise degrades hearing
intelligibility of human speech greatly. However, removing the babble without
creating artifacts in human speech is a challenging task in a low SNR
environment. Here, we sought to solve the problem by finding a `mapping'
between noisy speech spectra and clean speech spectra via supervised learning.
Specifically, we propose using fully Convolutional Neural Networks, which
consist of lesser number of parameters than fully connected networks. The
proposed network, Redundant Convolutional Encoder Decoder (R-CED), demonstrates
that a convolutional network can be 12 times smaller than a recurrent network
and yet achieves better performance, which shows its applicability for an
embedded system: the hearing aids.
Social Network Processes in the Isabelle and Coq Theorem Proving Communities
Authors: Jacques Fleuriot, Steven Obua, Phil Scott
Date: 2016-09-22 00:00:00
Tasks: Automated Theorem Proving

We identify the main actors in the Isabelle and Coq communities and describe
how they affect and influence their peers. This work explores selected
foundations of social networking analysis that we expect to be useful in the
context of the ProofPeer project, which is developing a new model for
interactive theorem proving based on collaboration and social interactions.
NdFluents: A Multi-dimensional Contexts Ontology
Authors: José M. Giménez-García, Antoine Zimmermann, Pierre Maret
Date: 2016-09-22 00:00:00

Annotating semantic data with metadata is becoming more and more important to
provide information about the statements being asserted. While initial
solutions proposed a data model to represent a specific dimension of
meta-information (such as time or provenance), the need for a general
annotation framework which allows representing different context dimensions is
needed. In this paper, we extend the 4dFluents ontology by Welty and Fikes---on
associating temporal validity to statements---to any dimension of context, and
discuss possible issues that multidimensional context representations have to
face and how we address them.
Learning Modular Neural Network Policies for Multi-Task and Multi-Robot Transfer
Authors: Coline Devin, Abhishek Gupta, Trevor Darrell, Pieter Abbeel, Sergey Levine
Date: 2016-09-22 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning, Zero-shot Generalization

Reinforcement learning (RL) can automate a wide variety of robotic skills,
but learning each new skill requires considerable real-world data collection
and manual representation engineering to design policy classes or features.
Using deep reinforcement learning to train general purpose neural network
policies alleviates some of the burden of manual representation engineering by
using expressive policy classes, but exacerbates the challenge of data
collection, since such methods tend to be less efficient than RL with
low-dimensional, hand-designed representations. Transfer learning can mitigate
this problem by enabling us to transfer information from one skill to another
and even from one robot to another. We show that neural network policies can be
decomposed into "task-specific" and "robot-specific" modules, where the
task-specific modules are shared across robots, and the robot-specific modules
are shared across all tasks on that robot. This allows for sharing task
information, such as perception, between robots and sharing robot information,
such as dynamics and kinematics, between tasks. We exploit this decomposition
to train mix-and-match modules that can solve new robot-task combinations that
were not seen during training. Using a novel neural network architecture, we
demonstrate the effectiveness of our transfer method for enabling zero-shot
generalization with a variety of robots and tasks in simulation for both visual
and non-visual tasks.
(Bandit) Convex Optimization with Biased Noisy Gradient Oracles
Authors: Xiaowei Hu, Prashanth L. A., András György, Csaba Szepesvári
Date: 2016-09-22 00:00:00

Algorithms for bandit convex optimization and online learning often rely on constructing noisy gradient estimates, which are then used in appropriately adjusted first-order algorithms, replacing actual gradients. Depending on the properties of the function to be optimized and the nature of ``noise'' in the bandit feedback, the bias and variance of gradient estimates exhibit various tradeoffs. In this paper we propose a novel framework that replaces the specific gradient estimation methods with an abstract oracle. With the help of the new framework we unify previous works, reproducing their results in a clean and concise fashion, while, perhaps more importantly, the framework also allows us to formally show that to achieve the optimal root-$n$ rate either the algorithms that use existing gradient estimators, or the proof techniques used to analyze them have to go beyond what exists today.
Knowledge Representation via Joint Learning of Sequential Text and Knowledge Graphs
Authors: Jiawei Wu, Ruobing Xie, Zhiyuan Liu, Maosong Sun
Date: 2016-09-22 00:00:00
Tasks: Informativeness, Knowledge Graphs, Link Prediction, Representation Learning, Sentence, Triple Classification

Textual information is considered as significant supplement to knowledge
representation learning (KRL). There are two main challenges for constructing
knowledge representations from plain texts: (1) How to take full advantages of
sequential contexts of entities in plain texts for KRL. (2) How to dynamically
select those informative sentences of the corresponding entities for KRL. In
this paper, we propose the Sequential Text-embodied Knowledge Representation
Learning to build knowledge representations from multiple sentences. Given each
reference sentence of an entity, we first utilize recurrent neural network with
pooling or long short-term memory network to encode the semantic information of
the sentence with respect to the entity. Then we further design an attention
model to measure the informativeness of each sentence, and build text-based
representations of entities. We evaluate our method on two tasks, including
triple classification and link prediction. Experimental results demonstrate
that our method outperforms other baselines on both tasks, which indicates that
our method is capable of selecting informative sentences and encoding the
textual information well into knowledge representations.
The Many-Body Expansion Combined with Neural Networks
Authors: Kun Yao, John E. Herr, John Parkhill
Date: 2016-09-22 00:00:00

Fragmentation methods such as the many-body expansion (MBE) are a common
strategy to model large systems by partitioning energies into a hierarchy of
decreasingly significant contributions. The number of fragments required for
chemical accuracy is still prohibitively expensive for ab-initio MBE to compete
with force field approximations for applications beyond single-point energies.
Alongside the MBE, empirical models of ab-initio potential energy surfaces have
improved, especially non-linear models based on neural networks (NN) which can
reproduce ab-initio potential energy surfaces rapidly and accurately. Although
they are fast, NNs suffer from their own curse of dimensionality; they must be
trained on a representative sample of chemical space. In this paper we examine
the synergy of the MBE and NN's, and explore their complementarity. The MBE
offers a systematic way to treat systems of arbitrary size and intelligently
sample chemical space. NN's reduce, by a factor in excess of $10^6$ the
computational overhead of the MBE and reproduce the accuracy of ab-initio
calculations without specialized force fields. We show they are remarkably
general, providing comparable accuracy with drastically different chemical
embeddings. To assess this we test a new chemical embedding which can be
inverted to predict molecules with desired properties.
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
Authors: Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, Yoshua Bengio
Date: 2016-09-22 00:00:00
Tasks: GPU

We introduce a method to train Quantized Neural Networks (QNNs) --- neural
networks with extremely low precision (e.g., 1-bit) weights and activations, at
run-time. At train-time the quantized weights and activations are used for
computing the parameter gradients. During the forward pass, QNNs drastically
reduce memory size and accesses, and replace most arithmetic operations with
bit-wise operations. As a result, power consumption is expected to be
drastically reduced. We trained QNNs over the MNIST, CIFAR-10, SVHN and
ImageNet datasets. The resulting QNNs achieve prediction accuracy comparable to
their 32-bit counterparts. For example, our quantized version of AlexNet with
1-bit weights and 2-bit activations achieves $51\%$ top-1 accuracy. Moreover,
we quantize the parameter gradients to 6-bits as well which enables gradients
computation using only bit-wise operation. Quantized recurrent neural networks
were tested over the Penn Treebank dataset, and achieved comparable accuracy as
their 32-bit counterparts using only 4-bits. Last but not least, we programmed
a binary matrix multiplication GPU kernel with which it is possible to run our
MNIST QNN 7 times faster than with an unoptimized GPU kernel, without suffering
any loss in classification accuracy. The QNN code is available online.