pwc-archive/papers-with-abstracts huggingface dataset:
Fuzzy Clustering Data Given in the Ordinal Scale
Authors: Zhengbing Hu, Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Viktoriia O. Samitova
Date: 2017-01-13 00:00:00
Tasks: Clustering

A fuzzy clustering algorithm for multidimensional data is proposed in this
article. The data is described by vectors whose components are linguistic
variables defined in an ordinal scale. The obtained results confirm the
efficiency of the proposed approach.
Learning the Number of Neurons in Deep Networks
Authors: Jose M. Alvarez, Mathieu Salzmann
Date: 2016-11-19 00:00:00
Proceeding: NeurIPS 2016 12

Nowadays, the number of layers and of neurons in each layer of a deep network
are typically set manually. While very deep and wide networks have proven
effective in general, they come at a high memory and computation cost, thus
making them impractical for constrained platforms. These networks, however, are
known to have many redundant parameters, and could thus, in principle, be
replaced by more compact architectures. In this paper, we introduce an approach
to automatically determining the number of neurons in each layer of a deep
network during learning. To this end, we propose to make use of structured
sparsity during learning. More precisely, we use a group sparsity regularizer
on the parameters of the network, where each group is defined to act on a
single neuron. Starting from an overcomplete network, we show that our approach
can reduce the number of parameters by up to 80\% while retaining or even
improving the network accuracy.
A Digital Fuzzy Edge Detector for Color Images
Authors: Yuan-Hang Zhang, Xie Li, Jing-Yun Xiao
Date: 2017-01-12 00:00:00
Tasks: Edge Detection, Image Segmentation, Semantic Segmentation

Edge detection is a classic problem in the field of image processing, which
lays foundations for other tasks such as image segmentation. Conventionally,
this operation is performed using gradient operators such as the Roberts or
Sobel operator, which can discover local changes in intensity levels. These
operators, however, perform poorly on low contrast images. In this paper, we
propose an edge detector architecture for color images based on fuzzy theory
and the Sobel operator. First, the R, G and B channels are extracted from an
image and enhanced using fuzzy methods, in order to suppress noise and improve
the contrast between the background and the objects. The Sobel operator is then
applied to each of the channels, which are finally combined into an edge map of
the origin image. Experimental results obtained through an FPGA-based
implementation have proved the proposed method effective.
Cost-Effective Active Learning for Deep Image Classification
Authors: Keze Wang, Dongyu Zhang, Ya Li, Ruimao Zhang, Liang Lin
Date: 2017-01-13 00:00:00
Tasks: Active Learning, Classification, Face Recognition, General Classification, image-classification, Image Classification, Incremental Learning, Object Categorization

Recent successes in learning-based image classification, however, heavily
rely on the large number of annotated training samples, which may require
considerable human efforts. In this paper, we propose a novel active learning
framework, which is capable of building a competitive classifier with optimal
feature representation via a limited amount of labeled training instances in an
incremental learning manner. Our approach advances the existing active learning
methods in two aspects. First, we incorporate deep convolutional neural
networks into active learning. Through the properly designed framework, the
feature representation and the classifier can be simultaneously updated with
progressively annotated informative samples. Second, we present a
cost-effective sample selection strategy to improve the classification
performance with less manual annotations. Unlike traditional methods focusing
on only the uncertain samples of low prediction confidence, we especially
discover the large amount of high confidence samples from the unlabeled set for
feature learning. Specifically, these high confidence samples are automatically
selected and iteratively assigned pseudo-labels. We thus call our framework
"Cost-Effective Active Learning" (CEAL) standing for the two
advantages.Extensive experiments demonstrate that the proposed CEAL framework
can achieve promising results on two challenging image classification datasets,
i.e., face recognition on CACD database [1] and object categorization on
Caltech-256 [2].
Bayesian System Identification based on Hierarchical Sparse Bayesian Learning and Gibbs Sampling with Application to Structural Damage Assessment
Authors: Yong Huang, James L. Beck, Hui Li
Date: 2017-01-13 00:00:00
Tasks: Bayesian Inference

The focus in this paper is Bayesian system identification based on noisy
incomplete modal data where we can impose spatially-sparse stiffness changes
when updating a structural model. To this end, based on a similar hierarchical
sparse Bayesian learning model from our previous work, we propose two Gibbs
sampling algorithms. The algorithms differ in their strategies to deal with the
posterior uncertainty of the equation-error precision parameter, but both
sample from the conditional posterior probability density functions (PDFs) for
the structural stiffness parameters and system modal parameters. The effective
dimension for the Gibbs sampling is low because iterative sampling is done from
only three conditional posterior PDFs that correspond to three parameter
groups, along with sampling of the equation-error precision parameter from
another conditional posterior PDF in one of the algorithms where it is not
integrated out as a "nuisance" parameter. A nice feature from a computational
perspective is that it is not necessary to solve a nonlinear eigenvalue problem
of a structural model. The effectiveness and robustness of the proposed
algorithms are illustrated by applying them to the IASE-ASCE Phase II simulated
and experimental benchmark studies. The goal is to use incomplete modal data
identified before and after possible damage to detect and assess
spatially-sparse stiffness reductions induced by any damage. Our past and
current focus on meeting challenges arising from Bayesian inference of
structural stiffness serve to strengthen the capability of vibration-based
structural system identification but our methods also have much broader
applicability for inverse problems in science and technology where system
matrices are to be inferred from noisy partial information about their
eigenquantities.
Bayesian Non-Homogeneous Markov Models via Polya-Gamma Data Augmentation with Applications to Rainfall Modeling
Authors: Tracy Holsclaw, Arthur M. Greene, Andrew W. Robertson, Padhraic Smyth
Date: 2017-01-11 00:00:00
Tasks: Data Augmentation, speech-recognition, Speech Recognition

Discrete-time hidden Markov models are a broadly useful class of
latent-variable models with applications in areas such as speech recognition,
bioinformatics, and climate data analysis. It is common in practice to
introduce temporal non-homogeneity into such models by making the transition
probabilities dependent on time-varying exogenous input variables via a
multinomial logistic parametrization. We extend such models to introduce
additional non-homogeneity into the emission distribution using a generalized
linear model (GLM), with data augmentation for sampling-based inference.
However, the presence of the logistic function in the state transition model
significantly complicates parameter inference for the overall model,
particularly in a Bayesian context. To address this we extend the
recently-proposed Polya-Gamma data augmentation approach to handle
non-homogeneous hidden Markov models (NHMMs), allowing the development of an
efficient Markov chain Monte Carlo (MCMC) sampling scheme. We apply our model
and inference scheme to 30 years of daily rainfall in India, leading to a
number of insights into rainfall-related phenomena in the region. Our proposed
approach allows for fully Bayesian analysis of relatively complex NHMMs on a
scale that was not possible with previous methods. Software implementing the
methods described in the paper is available via the R package NHMM.
Modeling Industrial ADMET Data with Multitask Networks
Authors: Steven Kearnes, Brian Goldman, Vijay Pande
Date: 2016-06-28 00:00:00
Tasks: Drug Discovery

Deep learning methods such as multitask neural networks have recently been
applied to ligand-based virtual screening and other drug discovery
applications. Using a set of industrial ADMET datasets, we compare neural
networks to standard baseline models and analyze multitask learning effects
with both random cross-validation and a more relevant temporal validation
scheme. We confirm that multitask learning can provide modest benefits over
single-task models and show that smaller datasets tend to benefit more than
larger datasets from multitask learning. Additionally, we find that adding
massive amounts of side information is not guaranteed to improve performance
relative to simpler multitask learning. Our results emphasize that multitask
effects are highly dataset-dependent, suggesting the use of dataset-specific
models to maximize overall performance.
An OpenCL(TM) Deep Learning Accelerator on Arria 10
Authors: Utku Aydonat, Shane O'Connell, Davor Capalija, Andrew C. Ling, Gordon R. Chiu
Date: 2017-01-13 00:00:00
Tasks: Deep Learning, GPU, image-classification, Image Classification

Convolutional neural nets (CNNs) have become a practical means to perform
vision tasks, particularly in the area of image classification. FPGAs are well
known to be able to perform convolutions efficiently, however, most recent
efforts to run CNNs on FPGAs have shown limited advantages over other devices
such as GPUs. Previous approaches on FPGAs have often been memory bound due to
the limited external memory bandwidth on the FPGA device. We show a novel
architecture written in OpenCL(TM), which we refer to as a Deep Learning
Accelerator (DLA), that maximizes data reuse and minimizes external memory
bandwidth. Furthermore, we show how we can use the Winograd transform to
significantly boost the performance of the FPGA. As a result, when running our
DLA on Intel's Arria 10 device we can achieve a performance of 1020 img/s, or
23 img/s/W when running the AlexNet CNN benchmark. This comes to 1382 GFLOPs
and is 10x faster with 8.4x more GFLOPS and 5.8x better efficiency than the
state-of-the-art on FPGAs. Additionally, 23 img/s/W is competitive against the
best publicly known implementation of AlexNet on nVidia's TitanX GPU.
Neural Combinatorial Optimization with Reinforcement Learning
Authors: Irwan Bello, Hieu Pham, Quoc V. Le, Mohammad Norouzi, Samy Bengio
Date: 2016-11-29 00:00:00
Tasks: Combinatorial Optimization, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Traveling Salesman Problem

This paper presents a framework to tackle combinatorial optimization problems
using neural networks and reinforcement learning. We focus on the traveling
salesman problem (TSP) and train a recurrent network that, given a set of city
coordinates, predicts a distribution over different city permutations. Using
negative tour length as the reward signal, we optimize the parameters of the
recurrent network using a policy gradient method. We compare learning the
network parameters on a set of training graphs against learning them on
individual test graphs. Despite the computational expense, without much
engineering and heuristic designing, Neural Combinatorial Optimization achieves
close to optimal results on 2D Euclidean graphs with up to 100 nodes. Applied
to the KnapSack, another NP-hard problem, the same method obtains optimal
solutions for instances with up to 200 items.
A Primer on Coordinate Descent Algorithms
Authors: Hao-Jun Michael Shi, Shenyinying Tu, Yangyang Xu, Wotao Yin
Date: 2016-09-30 00:00:00
Tasks: BIG-bench Machine Learning, compressed sensing, Distributed Computing

This monograph presents a class of algorithms called coordinate descent
algorithms for mathematicians, statisticians, and engineers outside the field
of optimization. This particular class of algorithms has recently gained
popularity due to their effectiveness in solving large-scale optimization
problems in machine learning, compressed sensing, image processing, and
computational statistics. Coordinate descent algorithms solve optimization
problems by successively minimizing along each coordinate or coordinate
hyperplane, which is ideal for parallelized and distributed computing. Avoiding
detailed technicalities and proofs, this monograph gives relevant theory and
examples for practitioners to effectively apply coordinate descent to modern
problems in data science and engineering.
Scalable, Trie-based Approximate Entity Extraction for Real-Time Financial Transaction Screening
Authors: Emrah Budur
Date: 2017-01-12 00:00:00
Tasks: Entity Extraction using GAN

Financial institutions have to screen their transactions to ensure that they
are not affiliated with terrorism entities. Developing appropriate solutions to
detect such affiliations precisely while avoiding any kind of interruption to
large amount of legitimate transactions is essential. In this paper, we present
building blocks of a scalable solution that may help financial institutions to
build their own software to extract terrorism entities out of both structured
and unstructured financial messages in real time and with approximate
similarity matching approach.
Simplified Minimal Gated Unit Variations for Recurrent Neural Networks
Authors: Joel Heck, Fathi M. Salem
Date: 2017-01-12 00:00:00

Recurrent neural networks with various types of hidden units have been used
to solve a diverse range of problems involving sequence data. Two of the most
recent proposals, gated recurrent units (GRU) and minimal gated units (MGU),
have shown comparable promising results on example public datasets. In this
paper, we introduce three model variants of the minimal gated unit (MGU) which
further simplify that design by reducing the number of parameters in the
forget-gate dynamic equation. These three model variants, referred to simply as
MGU1, MGU2, and MGU3, were tested on sequences generated from the MNIST dataset
and from the Reuters Newswire Topics (RNT) dataset. The new models have shown
similar accuracy to the MGU model while using fewer parameters and thus
lowering training expense. One model variant, namely MGU2, performed better
than MGU on the datasets considered, and thus may be used as an alternate to
MGU or GRU in recurrent neural networks.
Manifold Alignment Determination: finding correspondences across different data views
Authors: Andreas Damianou, Neil D. Lawrence, Carl Henrik Ek
Date: 2017-01-12 00:00:00

We present Manifold Alignment Determination (MAD), an algorithm for learning
alignments between data points from multiple views or modalities. The approach
is capable of learning correspondences between views as well as correspondences
between individual data-points. The proposed method requires only a few aligned
examples from which it is capable to recover a global alignment through a
probabilistic model. The strong, yet flexible regularization provided by the
generative model is sufficient to align the views. We provide experiments on
both synthetic and real data to highlight the benefit of the proposed approach.
Simplified Gating in Long Short-term Memory (LSTM) Recurrent Neural Networks
Authors: Yuzhen Lu, Fathi M. Salem
Date: 2017-01-12 00:00:00

The standard LSTM recurrent neural networks while very powerful in long-range
dependency sequence applications have highly complex structure and relatively
large (adaptive) parameters. In this work, we present empirical comparison
between the standard LSTM recurrent neural network architecture and three new
parameter-reduced variants obtained by eliminating combinations of the input
signal, bias, and hidden unit signals from individual gating signals. The
experiments on two sequence datasets show that the three new variants, called
simply as LSTM1, LSTM2, and LSTM3, can achieve comparable performance to the
standard LSTM model with less (adaptive) parameters.
Comprehension-guided referring expressions
Authors: Ruotian Luo, Gregory Shakhnarovich
Date: 2017-01-12 00:00:00
Proceeding: CVPR 2017 7
Tasks: Referring Expression, Referring expression generation

We consider generation and comprehension of natural language referring
expression for objects in an image. Unlike generic "image captioning" which
lacks natural standard evaluation criteria, quality of a referring expression
may be measured by the receiver's ability to correctly infer which object is
being described. Following this intuition, we propose two approaches to utilize
models trained for comprehension task to generate better expressions. First, we
use a comprehension module trained on human-generated expressions, as a
"critic" of referring expression generator. The comprehension module serves as
a differentiable proxy of human evaluation, providing training signal to the
generation module. Second, we use the comprehension module in a
generate-and-rerank pipeline, which chooses from candidate expressions
generated by a model according to their performance on the comprehension task.
We show that both approaches lead to improved referring expression generation
on multiple benchmark datasets.
SMARTies: Sentiment Models for Arabic Target Entities
Authors: Noura Farra, Kathleen McKeown
Date: 2017-01-12 00:00:00
Proceeding: EACL 2017 4
Tasks: Articles, Sentiment Analysis

We consider entity-level sentiment analysis in Arabic, a morphologically rich
language with increasing resources. We present a system that is applied to
complex posts written in response to Arabic newspaper articles. Our goal is to
identify important entity "targets" within the post along with the polarity
expressed about each target. We achieve significant improvements over multiple
baselines, demonstrating that the use of specific morphological representations
improves the performance of identifying both important targets and their
sentiment, and that the use of distributional semantic clusters further boosts
performances for these representations, especially when richer linguistic
resources are not available.
Generalized Inverse Classification
Authors: Michael T. Lash, Qihang Lin, W. Nick Street, Jennifer G. Robinson, Jeffrey Ohlmann
Date: 2016-10-05 00:00:00
Tasks: Classification, General Classification

Inverse classification is the process of perturbing an instance in a
meaningful way such that it is more likely to conform to a specific class.
Historical methods that address such a problem are often framed to leverage
only a single classifier, or specific set of classifiers. These works are often
accompanied by naive assumptions. In this work we propose generalized inverse
classification (GIC), which avoids restricting the classification model that
can be used. We incorporate this formulation into a refined framework in which
GIC takes place. Under this framework, GIC operates on features that are
immediately actionable. Each change incurs an individual cost, either linear or
non-linear. Such changes are subjected to occur within a specified level of
cumulative change (budget). Furthermore, our framework incorporates the
estimation of features that change as a consequence of direct actions taken
(indirectly changeable features). To solve such a problem, we propose three
real-valued heuristic-based methods and two sensitivity analysis-based
comparison methods, each of which is evaluated on two freely available
real-world datasets. Our results demonstrate the validity and benefits of our
formulation, framework, and methods.
Automatic Discoveries of Physical and Semantic Concepts via Association Priors of Neuron Groups
Authors: Shuai Li, Kui Jia, Xiaogang Wang
Date: 2016-12-30 00:00:00

The recent successful deep neural networks are largely trained in a
supervised manner. It {\it associates} complex patterns of input samples with
neurons in the last layer, which form representations of {\it concepts}. In
spite of their successes, the properties of complex patterns associated a
learned concept remain elusive. In this work, by analyzing how neurons are
associated with concepts in supervised networks, we hypothesize that with
proper priors to regulate learning, neural networks can automatically associate
neurons in the intermediate layers with concepts that are aligned with real
world concepts, when trained only with labels that associate concepts with top
level neurons, which is a plausible way for unsupervised learning. We develop a
prior to verify the hypothesis and experimentally find the proposed prior help
neural networks automatically learn both basic physical concepts at the lower
layers, e.g., rotation of filters, and highly semantic concepts at the higher
layers, e.g., fine-grained categories of an entry-level category.
Joint Dictionary Learning for Example-based Image Super-resolution
Authors: Mojtaba Sahraee-Ardakan, Mohsen Joneidi
Date: 2017-01-12 00:00:00
Tasks: Dictionary Learning, Image Super-Resolution, Super-Resolution

In this paper, we propose a new joint dictionary learning method for
example-based image super-resolution (SR), using sparse representation. The
low-resolution (LR) dictionary is trained from a set of LR sample image
patches. Using the sparse representation coefficients of these LR patches over
the LR dictionary, the high-resolution (HR) dictionary is trained by minimizing
the reconstruction error of HR sample patches. The error criterion used here is
the mean square error. In this way we guarantee that the HR patches have the
same sparse representation over HR dictionary as the LR patches over the LR
dictionary, and at the same time, these sparse representations can well
reconstruct the HR patches. Simulation results show the effectiveness of our
method compared to the state-of-art SR algorithms.
Improving Sampling from Generative Autoencoders with Markov Chains
Authors: Antonia Creswell, Kai Arulkumaran, Anil Anthony Bharath
Date: 2016-10-28 00:00:00

We focus on generative autoencoders, such as variational or adversarial
autoencoders, which jointly learn a generative model alongside an inference
model. Generative autoencoders are those which are trained to softly enforce a
prior on the latent distribution learned by the inference model. We call the
distribution to which the inference model maps observed samples, the learned
latent distribution, which may not be consistent with the prior. We formulate a
Markov chain Monte Carlo (MCMC) sampling process, equivalent to iteratively
decoding and encoding, which allows us to sample from the learned latent
distribution. Since, the generative model learns to map from the learned latent
distribution, rather than the prior, we may use MCMC to improve the quality of
samples drawn from the generative model, especially when the learned latent
distribution is far from the prior. Using MCMC sampling, we are able to reveal
previously unseen differences between generative autoencoders trained either
with or without a denoising criterion.
EgoSampling: Wide View Hyperlapse from Egocentric Videos
Authors: Tavi Halperin, Yair Poleg, Chetan Arora, Shmuel Peleg
Date: 2016-04-26 00:00:00

The possibility of sharing one's point of view makes use of wearable cameras
compelling. These videos are often long, boring and coupled with extreme shake,
as the camera is worn on a moving person. Fast forwarding (i.e. frame sampling)
is a natural choice for quick video browsing. However, this accentuates the
shake caused by natural head motion in an egocentric video, making the fast
forwarded video useless. We propose EgoSampling, an adaptive frame sampling
that gives stable, fast forwarded, hyperlapse videos. Adaptive frame sampling
is formulated as an energy minimization problem, whose optimal solution can be
found in polynomial time. We further turn the camera shake from a drawback into
a feature, enabling the increase in field-of-view of the output video. This is
obtained when each output frame is mosaiced from several input frames. The
proposed technique also enables the generation of a single hyperlapse video
from multiple egocentric videos, allowing even faster video consumption.