pwc-archive/papers-with-abstracts huggingface dataset:
A Novel Multi-clustering Method for Hierarchical Clusterings, Based on Boosting
Authors: Elaheh Rashedi, Abdolreza Mirzaei
Date: 2018-05-29 00:00:00
Tasks: Clustering

Bagging and boosting are proved to be the best methods of building multiple
classifiers in classification combination problems. In the area of "flat
clustering" problems, it is also recognized that multi-clustering methods based
on boosting provide clusterings of an improved quality. In this paper, we
introduce a novel multi-clustering method for "hierarchical clusterings" based
on boosting theory, which creates a more stable hierarchical clustering of a
dataset. The proposed algorithm includes a boosting iteration in which a
bootstrap of samples is created by weighted random sampling of elements from
the original dataset. A hierarchical clustering algorithm is then applied to
selected subsample to build a dendrogram which describes the hierarchy.
Finally, dissimilarity description matrices of multiple dendrogram results are
combined to a consensus one, using a hierarchical-clustering-combination
approach. Experiments on real popular datasets show that boosted method
provides superior quality solutions compared to standard hierarchical
clustering methods.
Depth and nonlinearity induce implicit exploration for RL
Authors: Justas Dauparas, Ryota Tomioka, Katja Hofmann
Date: 2018-05-29 00:00:00
Tasks: Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

The question of how to explore, i.e., take actions with uncertain outcomes to
learn about possible future rewards, is a key question in reinforcement
learning (RL). Here, we show a surprising result: We show that Q-learning with
nonlinear Q-function and no explicit exploration (i.e., a purely greedy policy)
can learn several standard benchmark tasks, including mountain car, equally
well as, or better than, the most commonly-used $\epsilon$-greedy exploration.
We carefully examine this result and show that both the depth of the Q-network
and the type of nonlinearity are important to induce such deterministic
exploration.
Active and Adaptive Sequential learning
Authors: Yuheng Bu, Jiaxun Lu, Venugopal V. Veeravalli
Date: 2018-05-29 00:00:00
Tasks: Active Learning

A framework is introduced for actively and adaptively solving a sequence of
machine learning problems, which are changing in bounded manner from one time
step to the next. An algorithm is developed that actively queries the labels of
the most informative samples from an unlabeled data pool, and that adapts to
the change by utilizing the information acquired in the previous steps. Our
analysis shows that the proposed active learning algorithm based on stochastic
gradient descent achieves a near-optimal excess risk performance for maximum
likelihood estimation. Furthermore, an estimator of the change in the learning
problems using the active learning samples is constructed, which provides an
adaptive sample size selection rule that guarantees the excess risk is bounded
for sufficiently large number of time steps. Experiments with synthetic and
real data are presented to validate our algorithm and theoretical results.
Supervised Policy Update for Deep Reinforcement Learning
Authors: Quan Vuong, Yiming Zhang, Keith W. Ross
Date: 2018-05-29 00:00:00
Proceeding: ICLR 2019
Tasks: Deep Reinforcement Learning, MuJoCo, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We propose a new sample-efficient methodology, called Supervised Policy
Update (SPU), for deep reinforcement learning. Starting with data generated by
the current policy, SPU formulates and solves a constrained optimization
problem in the non-parameterized proximal policy space. Using supervised
regression, it then converts the optimal non-parameterized policy to a
parameterized policy, from which it draws new samples. The methodology is
general in that it applies to both discrete and continuous action spaces, and
can handle a wide variety of proximity constraints for the non-parameterized
optimization problem. We show how the Natural Policy Gradient and Trust Region
Policy Optimization (NPG/TRPO) problems, and the Proximal Policy Optimization
(PPO) problem can be addressed by this methodology. The SPU implementation is
much simpler than TRPO. In terms of sample efficiency, our extensive
experiments show SPU outperforms TRPO in Mujoco simulated robotic tasks and
outperforms PPO in Atari video game tasks.
Deep Semantic Architecture with discriminative feature visualization for neuroimage analysis
Authors: Arna Ghosh, Fabien dal Maso, Marc Roig, Georgios D Mitsis, Marie-Hélène Boudrias
Date: 2018-05-29 00:00:00
Tasks: EEG, Electroencephalogram (EEG), feature selection

Neuroimaging data analysis often involves \emph{a-priori} selection of data
features to study the underlying neural activity. Since this could lead to
sub-optimal feature selection and thereby prevent the detection of subtle
patterns in neural activity, data-driven methods have recently gained
popularity for optimizing neuroimaging data analysis pipelines and thereby,
improving our understanding of neural mechanisms. In this context, we developed
a deep convolutional architecture that can identify discriminating patterns in
neuroimaging data and applied it to electroencephalography (EEG) recordings
collected from 25 subjects performing a hand motor task before and after a rest
period or a bout of exercise. The deep network was trained to classify subjects
into exercise and control groups based on differences in their EEG signals.
Subsequently, we developed a novel method termed the cue-combination for Class
Activation Map (ccCAM), which enabled us to identify discriminating
spatio-temporal features within definite frequency bands (23--33 Hz) and assess
the effects of exercise on the brain. Additionally, the proposed architecture
allowed the visualization of the differences in the propagation of underlying
neural activity across the cortex between the two groups, for the first time in
our knowledge. Our results demonstrate the feasibility of using deep network
architectures for neuroimaging analysis in different contexts such as, for the
identification of robust brain biomarkers to better characterize and
potentially treat neurological disorders.
Channel Gating Neural Networks
Authors: Weizhe Hua, Yuan Zhou, Christopher De Sa, Zhiru Zhang, G. Edward Suh
Date: 2018-05-29 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Knowledge Distillation, Network Pruning

This paper introduces channel gating, a dynamic, fine-grained, and hardware-efficient pruning scheme to reduce the computation cost for convolutional neural networks (CNNs). Channel gating identifies regions in the features that contribute less to the classification result, and skips the computation on a subset of the input channels for these ineffective regions. Unlike static network pruning, channel gating optimizes CNN inference at run-time by exploiting input-specific characteristics, which allows substantially reducing the compute cost with almost no accuracy loss. We experimentally show that applying channel gating in state-of-the-art networks achieves 2.7-8.0$\times$ reduction in floating-point operations (FLOPs) and 2.0-4.4$\times$ reduction in off-chip memory accesses with a minimal accuracy loss on CIFAR-10. Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2.6$\times$ without accuracy drop on ImageNet. We further demonstrate that channel gating can be realized in hardware efficiently. Our approach exhibits sparsity patterns that are well-suited to dense systolic arrays with minimal additional hardware. We have designed an accelerator for channel gating networks, which can be implemented using either FPGAs or ASICs. Running a quantized ResNet-18 model for ImageNet, our accelerator achieves an encouraging speedup of 2.4$\times$ on average, with a theoretical FLOP reduction of 2.8$\times$.
Can DNNs Learn to Lipread Full Sentences?
Authors: George Sterpu, Christian Saam, Naomi Harte
Date: 2018-05-29 00:00:00
Tasks: General Classification, Language Modeling, Language Modelling, Lipreading

Finding visual features and suitable models for lipreading tasks that are
more complex than a well-constrained vocabulary has proven challenging. This
paper explores state-of-the-art Deep Neural Network architectures for
lipreading based on a Sequence to Sequence Recurrent Neural Network. We report
results for both hand-crafted and 2D/3D Convolutional Neural Network visual
front-ends, online monotonic attention, and a joint Connectionist Temporal
Classification-Sequence-to-Sequence loss. The system is evaluated on the
publicly available TCD-TIMIT dataset, with 59 speakers and a vocabulary of over
6000 words. Results show a major improvement on a Hidden Markov Model
framework. A fuller analysis of performance across visemes demonstrates that
the network is not only learning the language model, but actually learning to
lipread.
Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings
Authors: Haw-Shiuan Chang, Amol Agrawal, Ananya Ganesh, Anirudha Desai, Vinayak Mathur, Alfred Hough, Andrew McCallum
Date: 2018-04-09 00:00:00
Proceeding: WS 2018 6
Tasks: Word Sense Induction

Word sense induction (WSI), which addresses polysemy by unsupervised
discovery of multiple word senses, resolves ambiguities for downstream NLP
tasks and also makes word representations more interpretable. This paper
proposes an accurate and efficient graph-based method for WSI that builds a
global non-negative vector embedding basis (which are interpretable like
topics) and clusters the basis indexes in the ego network of each polysemous
word. By adopting distributional inclusion vector embeddings as our basis
formation model, we avoid the expensive step of nearest neighbor search that
plagues other graph-based methods without sacrificing the quality of sense
clusters. Experiments on three datasets show that our proposed method produces
similar or better sense clusters and embeddings compared with previous
state-of-the-art methods while being significantly more efficient.
Probabilistic Trajectory Segmentation by Means of Hierarchical Dirichlet Process Switching Linear Dynamical Systems
Authors: Maximilian Sieb, Matthias Schultheis, Sebastian Szelag, Rudolf Lioutikov, Jan Peters
Date: 2018-05-29 00:00:00
Tasks: Segmentation

Using movement primitive libraries is an effective means to enable robots to solve more complex tasks. In order to build these movement libraries, current algorithms require a prior segmentation of the demonstration trajectories. A promising approach is to model the trajectory as being generated by a set of Switching Linear Dynamical Systems and inferring a meaningful segmentation by inspecting the transition points characterized by the switching dynamics. With respect to the learning, a nonparametric Bayesian approach is employed utilizing a Gibbs sampler.
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection
Authors: Haw-Shiuan Chang, ZiYun Wang, Luke Vilnis, Andrew McCallum
Date: 2017-10-02 00:00:00
Proceeding: NAACL 2018 6
Tasks: Hypernym Discovery, Question Answering, Relation Extraction

Modeling hypernymy, such as poodle is-a dog, is an important generalization
aid to many NLP tasks, such as entailment, coreference, relation extraction,
and question answering. Supervised learning from labeled hypernym sources, such
as WordNet, limits the coverage of these models, which can be addressed by
learning hypernyms from unlabeled text. Existing unsupervised methods either do
not scale to large vocabularies or yield unacceptably poor accuracy. This paper
introduces distributional inclusion vector embedding (DIVE), a
simple-to-implement unsupervised method of hypernym discovery via per-word
non-negative vector embeddings which preserve the inclusion property of word
contexts in a low-dimensional and interpretable space. In experimental
evaluations more comprehensive than any previous literature of which we are
aware-evaluating on 11 datasets using multiple existing as well as newly
proposed scoring functions-we find that our method provides up to double the
precision of previous unsupervised embeddings, and the highest average
performance, using a much more compact word representation, and yielding many
new state-of-the-art results.
A Unified Particle-Optimization Framework for Scalable Bayesian Sampling
Authors: Changyou Chen, Ruiyi Zhang, Wenlin Wang, Bai Li, Liqun Chen
Date: 2018-05-29 00:00:00

There has been recent interest in developing scalable Bayesian sampling
methods such as stochastic gradient MCMC (SG-MCMC) and Stein variational
gradient descent (SVGD) for big-data analysis. A standard SG-MCMC algorithm
simulates samples from a discrete-time Markov chain to approximate a target
distribution, thus samples could be highly correlated, an undesired property
for SG-MCMC. In contrary, SVGD directly optimizes a set of particles to
approximate a target distribution, and thus is able to obtain good
approximations with relatively much fewer samples. In this paper, we propose a
principle particle-optimization framework based on Wasserstein gradient flows
to unify SG-MCMC and SVGD, and to allow new algorithms to be developed. Our
framework interprets SG-MCMC as particle optimization on the space of
probability measures, revealing a strong connection between SG-MCMC and SVGD.
The key component of our framework is several particle-approximate techniques
to efficiently solve the original partial differential equations on the space
of probability measures. Extensive experiments on both synthetic data and deep
neural networks demonstrate the effectiveness and efficiency of our framework
for scalable Bayesian sampling.
LSTMs Exploit Linguistic Attributes of Data
Authors: Nelson F. Liu, Omer Levy, Roy Schwartz, Chenhao Tan, Noah A. Smith
Date: 2018-05-29 00:00:00
Proceeding: WS 2018 7
Tasks: Memorization, Open-Ended Question Answering

While recurrent neural networks have found success in a variety of natural
language processing applications, they are general models of sequential data.
We investigate how the properties of natural language data affect an LSTM's
ability to learn a nonlinguistic task: recalling elements from its input. We
find that models trained on natural language data are able to recall tokens
from much longer sequences than models trained on non-language sequential data.
Furthermore, we show that the LSTM learns to solve the memorization task by
explicitly using a subset of its neurons to count timesteps in the input. We
hypothesize that the patterns and structure in natural language data enable
LSTMs to learn by providing approximate ways of reducing loss, but
understanding the effect of different training data on the learnability of
LSTMs remains an open question.
Teaching Meaningful Explanations
Authors: Noel C. F. Codella, Michael Hind, Karthikeyan Natesan Ramamurthy, Murray Campbell, Amit Dhurandhar, Kush R. Varshney, Dennis Wei, Aleksandra Mojsilovic
Date: 2018-05-29 00:00:00
Tasks: BIG-bench Machine Learning

The adoption of machine learning in high-stakes applications such as
healthcare and law has lagged in part because predictions are not accompanied
by explanations comprehensible to the domain user, who often holds the ultimate
responsibility for decisions and outcomes. In this paper, we propose an
approach to generate such explanations in which training data is augmented to
include, in addition to features and labels, explanations elicited from domain
users. A joint model is then learned to produce both labels and explanations
from the input features. This simple idea ensures that explanations are
tailored to the complexity expectations and domain knowledge of the consumer.
Evaluation spans multiple modeling techniques on a game dataset, a (visual)
aesthetics dataset, a chemical odor dataset and a Melanoma dataset showing that
our approach is generalizable across domains and algorithms. Results
demonstrate that meaningful explanations can be reliably taught to machine
learning algorithms, and in some cases, also improve modeling accuracy.
High Dimensional Robust Sparse Regression
Authors: Liu Liu, Yanyao Shen, Tianyang Li, Constantine Caramanis
Date: 2018-05-29 00:00:00
Tasks: regression, Vocal Bursts Intensity Prediction

We provide a novel -- and to the best of our knowledge, the first -- algorithm for high dimensional sparse regression with constant fraction of corruptions in explanatory and/or response variables. Our algorithm recovers the true sparse parameters with sub-linear sample complexity, in the presence of a constant fraction of arbitrary corruptions. Our main contribution is a robust variant of Iterative Hard Thresholding. Using this, we provide accurate estimators: when the covariance matrix in sparse regression is identity, our error guarantee is near information-theoretically optimal. We then deal with robust sparse regression with unknown structured covariance matrix. We propose a filtering algorithm which consists of a novel randomized outlier removal technique for robust sparse mean estimation that may be of interest in its own right: the filtering algorithm is flexible enough to deal with unknown covariance. Also, it is orderwise more efficient computationally than the ellipsoid algorithm. Using sub-linear sample complexity, our algorithm achieves the best known (and first) error guarantee. We demonstrate the effectiveness on large-scale sparse regression problems with arbitrary corruptions.
Why are Big Data Matrices Approximately Low Rank?
Authors: Madeleine Udell, Alex Townsend
Date: 2017-05-21 00:00:00
Tasks: Recommendation Systems, Topic Models

Matrices of (approximate) low rank are pervasive in data science, appearing
in recommender systems, movie preferences, topic models, medical records, and
genomics. While there is a vast literature on how to exploit low rank structure
in these datasets, there is less attention on explaining why the low rank
structure appears in the first place. Here, we explain the effectiveness of low
rank models in data science by considering a simple generative model for these
matrices: we suppose that each row or column is associated to a (possibly high
dimensional) bounded latent variable, and entries of the matrix are generated
by applying a piecewise analytic function to these latent variables. These
matrices are in general full rank. However, we show that we can approximate
every entry of an $m \times n$ matrix drawn from this model to within a fixed
absolute error by a low rank matrix whose rank grows as $\mathcal O(\log(m +
n))$. Hence any sufficiently large matrix from such a latent variable model can
be approximated, up to a small entrywise error, by a low rank matrix.
Deep Learning under Privileged Information Using Heteroscedastic Dropout
Authors: John Lambert, Ozan Sener, Silvio Savarese
Date: 2018-05-29 00:00:00
Proceeding: CVPR 2018 6
Tasks: Deep Learning, image-classification, Image Classification, Machine Translation, Translation

Unlike machines, humans learn through rapid, abstract model-building. The
role of a teacher is not simply to hammer home right or wrong answers, but
rather to provide intuitive comments, comparisons, and explanations to a pupil.
This is what the Learning Under Privileged Information (LUPI) paradigm
endeavors to model by utilizing extra knowledge only available during training.
We propose a new LUPI algorithm specifically designed for Convolutional Neural
Networks (CNNs) and Recurrent Neural Networks (RNNs). We propose to use a
heteroscedastic dropout (i.e. dropout with a varying variance) and make the
variance of the dropout a function of privileged information. Intuitively, this
corresponds to using the privileged information to control the uncertainty of
the model output. We perform experiments using CNNs and RNNs for the tasks of
image classification and machine translation. Our method significantly
increases the sample efficiency during learning, resulting in higher accuracy
with a large margin when the number of training examples is limited. We also
theoretically justify the gains in sample efficiency by providing a
generalization error bound decreasing with $O(\frac{1}{n})$, where $n$ is the
number of training examples, in an oracle case.
Recurrent Residual Convolutional Neural Network based on U-Net (R2U-Net) for Medical Image Segmentation
Authors: Md Zahangir Alom, Mahmudul Hasan, Chris Yakopcic, Tarek M. Taha, Vijayan K. Asari
Date: 2018-02-20 00:00:00
Tasks: image-classification, Image Classification, Image Segmentation, Lesion Segmentation, Lung Nodule Segmentation, Medical Image Classification, Medical Image Segmentation, Retinal Vessel Segmentation, Segmentation, Semantic Segmentation, Skin Cancer Segmentation

Deep learning (DL) based semantic segmentation methods have been providing
state-of-the-art performance in the last few years. More specifically, these
techniques have been successfully applied to medical image classification,
segmentation, and detection tasks. One deep learning technique, U-Net, has
become one of the most popular for these applications. In this paper, we
propose a Recurrent Convolutional Neural Network (RCNN) based on U-Net as well
as a Recurrent Residual Convolutional Neural Network (RRCNN) based on U-Net
models, which are named RU-Net and R2U-Net respectively. The proposed models
utilize the power of U-Net, Residual Network, as well as RCNN. There are
several advantages of these proposed architectures for segmentation tasks.
First, a residual unit helps when training deep architecture. Second, feature
accumulation with recurrent residual convolutional layers ensures better
feature representation for segmentation tasks. Third, it allows us to design
better U-Net architecture with same number of network parameters with better
performance for medical image segmentation. The proposed models are tested on
three benchmark datasets such as blood vessel segmentation in retina images,
skin cancer segmentation, and lung lesion segmentation. The experimental
results show superior performance on segmentation tasks compared to equivalent
models including U-Net and residual U-Net (ResU-Net).
Semantically-informed distance and similarity measures for paraphrase plagiarism identification
Authors: Miguel A. Álvarez-Carmona, Marc Franco-Salvador, Esaú Villatoro-Tello, Manuel Montes-y-Gómez, Paolo Rosso, Luis Villaseñor-Pineda
Date: 2018-05-29 00:00:00

Paraphrase plagiarism identification represents a very complex task given
that plagiarized texts are intentionally modified through several rewording
techniques. Accordingly, this paper introduces two new measures for evaluating
the relatedness of two given texts: a semantically-informed similarity measure
and a semantically-informed edit distance. Both measures are able to extract
semantic information from either an external resource or a distributed
representation of words, resulting in informative features for training a
supervised classifier for detecting paraphrase plagiarism. Obtained results
indicate that the proposed metrics are consistently good in detecting different
types of paraphrase plagiarism. In addition, results are very competitive
against state-of-the art methods having the advantage of representing a much
more simple but equally effective solution.
Automatic Identification of Arabic expressions related to future events in Lebanon's economy
Authors: Moustafa Al-Hajj, Amani Sabra
Date: 2018-05-29 00:00:00

In this paper, we propose a method to automatically identify future events in
Lebanon's economy from Arabic texts. Challenges are threefold: first, we need
to build a corpus of Arabic texts that covers Lebanon's economy; second, we
need to study how future events are expressed linguistically in these texts;
and third, we need to automatically identify the relevant textual segments
accordingly. We will validate this method on a constructed corpus form the web
and show that it has very promising results. To do so, we will be using SLCSAS,
a system for semantic analysis, based on the Contextual Explorer method, and
"AlKhalil Morpho Sys" system for morpho-syntactic analysis.