pwc-archive/papers-with-abstracts huggingface dataset:
Common-Description Learning: A Framework for Learning Algorithms and Generating Subproblems from Few Examples
Authors: Basem G. El-Barashy
Date: 2016-05-01 00:00:00

Current learning algorithms face many difficulties in learning simple
patterns and using them to learn more complex ones. They also require more
examples than humans do to learn the same pattern, assuming no prior knowledge.
In this paper, a new learning framework is introduced that is called
common-description learning (CDL). This framework has been tested on 32 small
multi-task datasets, and the results show that it was able to learn complex
algorithms from a few number of examples. The final model is perfectly
interpretable and its depth depends on the question. What is meant by depth
here is that whenever needed, the model learns to break down the problem into
simpler subproblems and solves them using previously learned models. Finally,
we explain the capabilities of our framework in discovering complex relations
in data and how it can help in improving language understanding in machines.
Text-mining the NeuroSynth corpus using Deep Boltzmann Machines
Authors: Ricardo Pio Monti, Romy Lorenz, Robert Leech, Christoforos Anagnostopoulos, Giovanni Montana
Date: 2016-05-01 00:00:00

Large-scale automated meta-analysis of neuroimaging data has recently
established itself as an important tool in advancing our understanding of human
brain function. This research has been pioneered by NeuroSynth, a database
collecting both brain activation coordinates and associated text across a large
cohort of neuroimaging research papers. One of the fundamental aspects of such
meta-analysis is text-mining. To date, word counts and more sophisticated
methods such as Latent Dirichlet Allocation have been proposed. In this work we
present an unsupervised study of the NeuroSynth text corpus using Deep
Boltzmann Machines (DBMs). The use of DBMs yields several advantages over the
aforementioned methods, principal among which is the fact that it yields both
word and document embeddings in a high-dimensional vector space. Such
embeddings serve to facilitate the use of traditional machine learning
techniques on the text corpus. The proposed DBM model is shown to learn
embeddings with a clear semantic structure.
Deep Learning for Surface Material Classification Using Haptic And Visual Information
Authors: Haitian Zheng, Lu Fang, Mengqi Ji, Matti Strese, Yigitcan Ozer, Eckehard Steinbach
Date: 2015-12-21 00:00:00
Tasks: Classification, Deep Learning, General Classification, Material Classification

When a user scratches a hand-held rigid tool across an object surface, an
acceleration signal can be captured, which carries relevant information about
the surface. More importantly, such a haptic signal is complementary to the
visual appearance of the surface, which suggests the combination of both
modalities for the recognition of the surface material. In this paper, we
present a novel deep learning method dealing with the surface material
classification problem based on a Fully Convolutional Network (FCN), which
takes as input the aforementioned acceleration signal and a corresponding image
of the surface texture. Compared to previous surface material classification
solutions, which rely on a careful design of hand-crafted domain-specific
features, our method automatically extracts discriminative features utilizing
the advanced deep learning methodologies. Experiments performed on the TUM
surface material database demonstrate that our method achieves state-of-the-art
classification accuracy robustly and efficiently.
Adaptive Concentration of Regression Trees, with Application to Random Forests
Authors: Stefan Wager, Guenther Walther
Date: 2015-03-22 00:00:00
Tasks: Model Selection, regression, valid

We study the convergence of the predictive surface of regression trees and
forests. To support our analysis we introduce a notion of adaptive
concentration for regression trees. This approach breaks tree training into a
model selection phase in which we pick the tree splits, followed by a model
fitting phase where we find the best regression model consistent with these
splits. We then show that the fitted regression tree concentrates around the
optimal predictor with the same splits: as d and n get large, the discrepancy
is with high probability bounded on the order of sqrt(log(d) log(n)/k)
uniformly over the whole regression surface, where d is the dimension of the
feature space, n is the number of training examples, and k is the minimum leaf
size for each tree. We also provide rate-matching lower bounds for this
adaptive concentration statement. From a practical perspective, our result
enables us to prove consistency results for adaptively grown forests in high
dimensions, and to carry out valid post-selection inference in the sense of
Berk et al. [2013] for subgroups defined by tree leaves.
Enforcing Template Representability and Temporal Consistency for Adaptive Sparse Tracking
Authors: Xue Yang, Fei Han, Hua Wang, Hao Zhang
Date: 2016-04-30 00:00:00
Tasks: Descriptive, Visual Tracking

Sparse representation has been widely studied in visual tracking, which has
shown promising tracking performance. Despite a lot of progress, the visual
tracking problem is still a challenging task due to appearance variations over
time. In this paper, we propose a novel sparse tracking algorithm that well
addresses temporal appearance changes, by enforcing template representability
and temporal consistency (TRAC). By modeling temporal consistency, our
algorithm addresses the issue of drifting away from a tracking target. By
exploring the templates' long-term-short-term representability, the proposed
method adaptively updates the dictionary using the most descriptive templates,
which significantly improves the robustness to target appearance changes. We
compare our TRAC algorithm against the state-of-the-art approaches on 12
challenging benchmark image sequences. Both qualitative and quantitative
results demonstrate that our algorithm significantly outperforms previous
state-of-the-art trackers.
Unsupervised and Semi-supervised Learning with Categorical Generative Adversarial Networks
Authors: Jost Tobias Springenberg
Date: 2015-11-19 00:00:00
Tasks: Clustering, General Classification, image-classification, Image Classification, Robust classification, Unsupervised Image Classification, Unsupervised MNIST

In this paper we present a method for learning a discriminative classifier
from unlabeled or partially labeled data. Our approach is based on an objective
function that trades-off mutual information between observed examples and their
predicted categorical class distribution, against robustness of the classifier
to an adversarial generative model. The resulting algorithm can either be
interpreted as a natural generalization of the generative adversarial networks
(GAN) framework or as an extension of the regularized information maximization
(RIM) framework to robust classification against an optimal adversary. We
empirically evaluate our method - which we dub categorical generative
adversarial networks (or CatGAN) - on synthetic data as well as on challenging
image classification tasks, demonstrating the robustness of the learned
classifiers. We further qualitatively assess the fidelity of samples generated
by the adversarial generator that is learned alongside the discriminative
classifier, and identify links between the CatGAN objective and discriminative
clustering algorithms (such as RIM).
Kernel Balancing: A flexible non-parametric weighting procedure for estimating causal effects
Authors: Chad Hazlett
Date: 2016-04-30 00:00:00

In the absence of unobserved confounders, matching and weighting methods are
widely used to estimate causal quantities including the Average Treatment
Effect on the Treated (ATT). Unfortunately, these methods do not necessarily
achieve their goal of making the multivariate distribution of covariates for
the control group identical to that of the treated, leaving some (potentially
multivariate) functions of the covariates with different means between the two
groups. When these "imbalanced" functions influence the non-treatment potential
outcome, the conditioning on observed covariates fails, and ATT estimates may
be biased. Kernel balancing, introduced here, targets a weaker requirement for
unbiased ATT estimation, specifically, that the expected non-treatment
potential outcome for the treatment and control groups are equal. The
conditional expectation of the non-treatment potential outcome is assumed to
fall in the space of functions associated with a choice of kernel, implying a
set of basis functions in which this regression surface is linear. Weights are
then chosen on the control units such that the treated and control group have
equal means on these basis functions. As a result, the expectation of the
non-treatment potential outcome must also be equal for the treated and control
groups after weighting, allowing unbiased ATT estimation by subsequent
difference in means or an outcome model using these weights. Moreover, the
weights produced are (1) precisely those that equalize a particular
kernel-based approximation of the multivariate distribution of covariates for
the treated and control, and (2) equivalent to a form of stabilized inverse
propensity score weighting, though it does not require assuming any model of
the treatment assignment mechanism. An R package, KBAL, is provided to
implement this approach.
Multi-Scale Context Aggregation by Dilated Convolutions
Authors: Fisher Yu, Vladlen Koltun
Date: 2015-11-23 00:00:00
Tasks: General Classification, image-classification, Prediction, Real-Time Semantic Segmentation, Segmentation, Semantic Segmentation

State-of-the-art models for semantic segmentation are based on adaptations of
convolutional networks that had originally been designed for image
classification. However, dense prediction and image classification are
structurally different. In this work, we develop a new convolutional network
module that is specifically designed for dense prediction. The presented module
uses dilated convolutions to systematically aggregate multi-scale contextual
information without losing resolution. The architecture is based on the fact
that dilated convolutions support exponential expansion of the receptive field
without loss of resolution or coverage. We show that the presented context
module increases the accuracy of state-of-the-art semantic segmentation
systems. In addition, we examine the adaptation of image classification
networks to dense prediction and show that simplifying the adapted network can
increase accuracy.
3D Keypoint Detection Based on Deep Neural Network with Sparse Autoencoder
Authors: Xinyu Lin, Ce Zhu, Qian Zhang, Yipeng Liu
Date: 2016-04-30 00:00:00
Tasks: Keypoint Detection, regression

Researchers have proposed various methods to extract 3D keypoints from the
surface of 3D mesh models over the last decades, but most of them are based on
geometric methods, which lack enough flexibility to meet the requirements for
various applications. In this paper, we propose a new method on the basis of
deep learning by formulating the 3D keypoint detection as a regression problem
using deep neural network (DNN) with sparse autoencoder (SAE) as our regression
model. Both local information and global information of a 3D mesh model in
multi-scale space are fully utilized to detect whether a vertex is a keypoint
or not. SAE can effectively extract the internal structure of these two kinds
of information and formulate high-level features for them, which is beneficial
to the regression model. Three SAEs are used to formulate the hidden layers of
the DNN and then a logistic regression layer is trained to process the
high-level features produced in the third SAE. Numerical experiments show that
the proposed DNN based 3D keypoint detection algorithm outperforms current five
state-of-the-art methods for various 3D mesh models.
An Improved System for Sentence-level Novelty Detection in Textual Streams
Authors: Xinyu Fu, Eugene Ch'ng, Uwe Aickelin, Lanyun Zhang
Date: 2016-04-30 00:00:00
Tasks: Event Detection, Novelty Detection, Sentence

Novelty detection in news events has long been a difficult problem. A number
of models performed well on specific data streams but certain issues are far
from being solved, particularly in large data streams from the WWW where
unpredictability of new terms requires adaptation in the vector space model. We
present a novel event detection system based on the Incremental Term
Frequency-Inverse Document Frequency (TF-IDF) weighting incorporated with
Locality Sensitive Hashing (LSH). Our system could efficiently and effectively
adapt to the changes within the data streams of any new terms with continual
updates to the vector space model. Regarding miss probability, our proposed
novelty detection framework outperforms a recognised baseline system by
approximately 16% when evaluating a benchmark dataset from Google News.
Computational Cost Reduction in Learned Transform Classifications
Authors: Emerson Lopes Machado, Cristiano Jacques Miosso, Ricardo von Borries, Murilo Coutinho, Pedro de Azevedo Berger, Thiago Marques, Ricardo Pezzuol Jacobi
Date: 2015-04-26 00:00:00
Tasks: General Classification

We present a theoretical analysis and empirical evaluations of a novel set of
techniques for computational cost reduction of classifiers that are based on
learned transform and soft-threshold. By modifying optimization procedures for
dictionary and classifier training, as well as the resulting dictionary
entries, our techniques allow to reduce the bit precision and to replace each
floating-point multiplication by a single integer bit shift. We also show how
the optimization algorithms in some dictionary training methods can be modified
to penalize higher-energy dictionaries. We applied our techniques with the
classifier Learning Algorithm for Soft-Thresholding, testing on the datasets
used in its original paper. Our results indicate it is feasible to use solely
sums and bit shifts of integers to classify at test time with a limited
reduction of the classification accuracy. These low power operations are a
valuable trade off in FPGA implementations as they increase the classification
throughput while decrease both energy consumption and manufacturing cost.
Application of artificial neural networks and genetic algorithms for crude fractional distillation process modeling
Authors: Lukasz Pater
Date: 2016-04-30 00:00:00

This work presents the application of the artificial neural networks, trained
and structurally optimized by genetic algorithms, for modeling of crude
distillation process at PKN ORLEN S.A. refinery. Models for the main
fractionator distillation column products were developed using historical data.
Quality of the fractions were predicted based on several chosen process
variables. The performance of the model was validated using test data. Neural
networks used in companion with genetic algorithms proved that they can
accurately predict fractions quality shifts, reproducing the results of the
standard laboratory analysis. Simple knowledge extraction method from neural
network model built was also performed. Genetic algorithms can be successfully
utilized in efficient training of large neural networks and finding their
optimal structures.
Constructive neural network learning
Authors: Shaobo Lin, Jinshan Zeng, Xiaoqin Zhang
Date: 2016-04-30 00:00:00

In this paper, we aim at developing scalable neural network-type learning
systems. Motivated by the idea of "constructive neural networks" in
approximation theory, we focus on "constructing" rather than "training"
feed-forward neural networks (FNNs) for learning, and propose a novel FNNs
learning system called the constructive feed-forward neural network (CFN).
Theoretically, we prove that the proposed method not only overcomes the
classical saturation problem for FNN approximation, but also reaches the
optimal learning rate when the regression function is smooth, while the
state-of-the-art learning rates established for traditional FNNs are only near
optimal (up to a logarithmic factor). A series of numerical simulations are
provided to show the efficiency and feasibility of CFN via comparing with the
well-known regularized least squares (RLS) with Gaussian kernel and extreme
learning machine (ELM).
Deep Colorization
Authors: Zezhou Cheng, Qingxiong Yang, Bin Sheng
Date: 2016-04-30 00:00:00
Proceeding: ICCV 2015 12
Tasks: Clustering, Colorization, Image Clustering, Patch Matching

This paper investigates into the colorization problem which converts a
grayscale image to a colorful version. This is a very difficult problem and
normally requires manual adjustment to achieve artifact-free quality. For
instance, it normally requires human-labelled color scribbles on the grayscale
target image or a careful selection of colorful reference images (e.g.,
capturing the same scene in the grayscale target image). Unlike the previous
methods, this paper aims at a high-quality fully-automatic colorization method.
With the assumption of a perfect patch matching technique, the use of an
extremely large-scale reference database (that contains sufficient color
images) is the most reliable solution to the colorization problem. However,
patch matching noise will increase with respect to the size of the reference
database in practice. Inspired by the recent success in deep learning
techniques which provide amazing modeling of large-scale data, this paper
re-formulates the colorization problem so that deep learning techniques can be
directly employed. To ensure artifact-free quality, a joint bilateral filtering
based post-processing step is proposed. We further develop an adaptive image
clustering technique to incorporate the global image information. Numerous
experiments demonstrate that our method outperforms the state-of-art algorithms
both in terms of quality and speed.
Higher Order Recurrent Neural Networks
Authors: Rohollah Soltani, Hui Jiang
Date: 2016-04-30 00:00:00
Tasks: Language Modeling, Language Modelling

In this paper, we study novel neural network structures to better model long
term dependency in sequential data. We propose to use more memory units to keep
track of more preceding states in recurrent neural networks (RNNs), which are
all recurrently fed to the hidden layers as feedback through different weighted
paths. By extending the popular recurrent structure in RNNs, we provide the
models with better short-term memory mechanism to learn long term dependency in
sequences. Analogous to digital filters in signal processing, we call these
structures as higher order RNNs (HORNNs). Similar to RNNs, HORNNs can also be
learned using the back-propagation through time method. HORNNs are generally
applicable to a variety of sequence modelling tasks. In this work, we have
examined HORNNs for the language modeling task using two popular data sets,
namely the Penn Treebank (PTB) and English text8 data sets. Experimental
results have shown that the proposed HORNNs yield the state-of-the-art
performance on both data sets, significantly outperforming the regular RNNs as
well as the popular LSTMs.
Visually Indicated Sounds
Authors: Andrew Owens, Phillip Isola, Josh Mcdermott, Antonio Torralba, Edward H. Adelson, William T. Freeman
Date: 2015-12-28 00:00:00
Proceeding: CVPR 2016 6

Objects make distinctive sounds when they are hit or scratched. These sounds
reveal aspects of an object's material properties, as well as the actions that
produced them. In this paper, we propose the task of predicting what sound an
object makes when struck as a way of studying physical interactions within a
visual scene. We present an algorithm that synthesizes sound from silent videos
of people hitting and scratching objects with a drumstick. This algorithm uses
a recurrent neural network to predict sound features from videos and then
produces a waveform from these features with an example-based synthesis
procedure. We show that the sounds predicted by our model are realistic enough
to fool participants in a "real or fake" psychophysical experiment, and that
they convey significant information about material properties and physical
interactions.
Distributed Cell Association for Energy Harvesting IoT Devices in Dense Small Cell Networks: A Mean-Field Multi-Armed Bandit Approach
Authors: Setareh Maghsudi, Ekram Hossain
Date: 2016-04-30 00:00:00

The emerging Internet of Things (IoT)-driven ultra-dense small cell networks
(UD-SCNs) will need to combat a variety of challenges. On one hand, massive
number of devices sharing the limited wireless resources will render
centralized control mechanisms infeasible due to the excessive cost of
information acquisition and computations. On the other hand, to reduce energy
consumption from fixed power grid and/or battery, many IoT devices may need to
depend on the energy harvested from the ambient environment (e.g., from RF
transmissions, environmental sources). However, due to the opportunistic nature
of energy harvesting, this will introduce uncertainty in the network operation.
In this article, we study the distributed cell association problem for energy
harvesting IoT devices in UD-SCNs. After reviewing the state-of-the-art
research on the cell association problem in small cell networks, we outline the
major challenges for distributed cell association in IoT-driven UD-SCNs where
the IoT devices will need to perform cell association in a distributed manner
in presence of uncertainty (e.g., limited knowledge on channel/network) and
limited computational capabilities. To this end, we propose an approach based
on mean-field multi-armed bandit games to solve the uplink cell association
problem for energy harvesting IoT devices in a UD-SCN. This approach is
particularly suitable to analyze large multi-agent systems under uncertainty
and lack of information. We provide some theoretical results as well as
preliminary performance evaluation results for the proposed approach.
DisturbLabel: Regularizing CNN on the Loss Layer
Authors: Lingxi Xie, Jingdong Wang, Zhen Wei, Meng Wang, Qi Tian
Date: 2016-04-30 00:00:00
Proceeding: CVPR 2016 6
Tasks: Data Augmentation

During a long period of time we are combating over-fitting in the CNN
training process with model regularization, including weight decay, model
averaging, data augmentation, etc. In this paper, we present DisturbLabel, an
extremely simple algorithm which randomly replaces a part of labels as
incorrect values in each iteration. Although it seems weird to intentionally
generate incorrect training labels, we show that DisturbLabel prevents the
network training from over-fitting by implicitly averaging over exponentially
many networks which are trained with different label sets. To the best of our
knowledge, DisturbLabel serves as the first work which adds noises on the loss
layer. Meanwhile, DisturbLabel cooperates well with Dropout to provide
complementary regularization functions. Experiments demonstrate competitive
recognition results on several popular image recognition datasets.
InterActive: Inter-Layer Activeness Propagation
Authors: Lingxi Xie, Liang Zheng, Jingdong Wang, Alan Yuille, Qi Tian
Date: 2016-04-30 00:00:00
Proceeding: CVPR 2016 6
Tasks: Descriptive, General Classification

An increasing number of computer vision tasks can be tackled with deep
features, which are the intermediate outputs of a pre-trained Convolutional
Neural Network. Despite the astonishing performance, deep features extracted
from low-level neurons are still below satisfaction, arguably because they
cannot access the spatial context contained in the higher layers. In this
paper, we present InterActive, a novel algorithm which computes the activeness
of neurons and network connections. Activeness is propagated through a neural
network in a top-down manner, carrying high-level context and improving the
descriptive power of low-level and mid-level neurons. Visualization indicates
that neuron activeness can be interpreted as spatial-weighted neuron responses.
We achieve state-of-the-art classification performance on a wide range of image
datasets.