pwc-archive/papers-with-abstracts huggingface dataset:
SAFE: Spectral Evolution Analysis Feature Extraction for Non-Stationary Time Series Prediction
Authors: Arief Koesdwiady, Fakhri Karray
Date: 2018-03-04 00:00:00
Tasks: GPU, Time Series, Time Series Analysis, Time Series Prediction

This paper presents a practical approach for detecting non-stationarity in
time series prediction. This method is called SAFE and works by monitoring the
evolution of the spectral contents of time series through a distance function.
This method is designed to work in combination with state-of-the-art machine
learning methods in real time by informing the online predictors to perform
necessary adaptation when a non-stationarity presents. We also propose an
algorithm to proportionally include some past data in the adaption process to
overcome the Catastrophic Forgetting problem. To validate our hypothesis and
test the effectiveness of our approach, we present comprehensive experiments in
different elements of the approach involving artificial and real-world
datasets. The experiments show that the proposed method is able to
significantly save computational resources in term of processor or GPU cycles
while maintaining high prediction performances.
New Results on Multi-Step Traffic Flow Prediction
Authors: Arief Koesdwiady, Fakhri Karray
Date: 2018-03-04 00:00:00
Tasks: Data Augmentation, Generative Adversarial Network, Prediction

In its simplest form, the traffic flow prediction problem is restricted to
predicting a single time-step into the future. Multi-step traffic flow
prediction extends this set-up to the case where predicting multiple time-steps
into the future based on some finite history is of interest. This problem is
significantly more difficult than its single-step variant and is known to
suffer from degradation in predictions as the time step increases. In this
paper, two approaches to improve multi-step traffic flow prediction performance
in recursive and multi-output settings are introduced. In particular, a model
that allows recursive prediction approaches to take into account the temporal
context in term of time-step index when making predictions is introduced. In
addition, a conditional generative adversarial network-based data augmentation
method is proposed to improve prediction performance in the multi-output
setting. The experiments on a real-world traffic flow dataset show that the two
methods improve on multi-step traffic flow prediction in recursive and
multi-output settings, respectively.
Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks
Authors: Sayeh Sharify, Alberto Delmas Lascorz, Kevin Siu, Patrick Judd, Andreas Moshovos
Date: 2017-06-23 00:00:00
Tasks: image-classification, Image Classification

Loom (LM), a hardware inference accelerator for Convolutional Neural Networks
(CNNs) is presented. In LM every bit of data precision that can be saved
translates to proportional performance gains. Specifically, for convolutional
layers LM's execution time scales inversely proportionally with the precisions
of both weights and activations. For fully-connected layers LM's performance
scales inversely proportionally with the precision of the weights. LM targets
area- and bandwidth-constrained System-on-a-Chip designs such as those found on
mobile devices that cannot afford the multi-megabyte buffers that would be
needed to store each layer on-chip. Accordingly, given a data bandwidth budget,
LM boosts energy efficiency and performance over an equivalent bit-parallel
accelerator. For both weights and activations LM can exploit profile-derived
perlayer precisions. However, at runtime LM further trims activation precisions
at a much smaller than a layer granularity. Moreover, it can naturally exploit
weight precision variability at a smaller granularity than a layer. On average,
across several image classification CNNs and for a configuration that can
perform the equivalent of 128 16b x 16b multiply-accumulate operations per
cycle LM outperforms a state-of-the-art bit-parallel accelerator [1] by 4.38x
without any loss in accuracy while being 3.54x more energy efficient. LM can
trade-off accuracy for additional improvements in execution performance and
energy efficiency and compares favorably to an accelerator that targeted only
activation precisions. We also study 2- and 4-bit LM variants and find the the
2-bit per cycle variant is the most energy efficient.
Support Vector Machine Active Learning Algorithms with Query-by-Committee versus Closest-to-Hyperplane Selection
Authors: Michael Bloodgood
Date: 2018-01-24 00:00:00
Tasks: Active Learning, General Classification, Relation Extraction, text-classification, Text Classification

This paper investigates and evaluates support vector machine active learning
algorithms for use with imbalanced datasets, which commonly arise in many
applications such as information extraction applications. Algorithms based on
closest-to-hyperplane selection and query-by-committee selection are combined
with methods for addressing imbalance such as positive amplification based on
prevalence statistics from initial random samples. Three algorithms (ClosestPA,
QBagPA, and QBoostPA) are presented and carefully evaluated on datasets for
text classification and relation extraction. The ClosestPA algorithm is shown
to consistently outperform the other two in a variety of ways and insights are
provided as to why this is the case.
QuaterNet: A Quaternion-based Recurrent Model for Human Motion
Authors: Dario Pavllo, David Grangier, Michael Auli
Date: 2018-05-16 00:00:00
Tasks: 3D Human Pose Estimation, Motion Estimation, Position

Deep learning for predicting or generating 3D human pose sequences is an
active research area. Previous work regresses either joint rotations or joint
positions. The former strategy is prone to error accumulation along the
kinematic chain, as well as discontinuities when using Euler angle or
exponential map parameterizations. The latter requires re-projection onto
skeleton constraints to avoid bone stretching and invalid configurations. This
work addresses both limitations. Our recurrent network, QuaterNet, represents
rotations with quaternions and our loss function performs forward kinematics on
a skeleton to penalize absolute position errors instead of angle errors. On
short-term predictions, QuaterNet improves the state-of-the-art quantitatively.
For long-term generation, our approach is qualitatively judged as realistic as
recent neural strategies from the graphics literature.
Impact of Batch Size on Stopping Active Learning for Text Classification
Authors: Garrett Beatty, Ethan Kochis, Michael Bloodgood
Date: 2018-01-24 00:00:00
Tasks: Active Learning, General Classification, Open-Ended Question Answering, text-classification, Text Classification

When using active learning, smaller batch sizes are typically more efficient
from a learning efficiency perspective. However, in practice due to speed and
human annotator considerations, the use of larger batch sizes is necessary.
While past work has shown that larger batch sizes decrease learning efficiency
from a learning curve perspective, it remains an open question how batch size
impacts methods for stopping active learning. We find that large batch sizes
degrade the performance of a leading stopping method over and above the
degradation that results from reduced learning efficiency. We analyze this
degradation and find that it can be mitigated by changing the window size
parameter of how many past iterations of learning are taken into account when
making the stopping decision. We find that when using larger batch sizes,
stopping methods are more effective when smaller window sizes are used.
Resisting Large Data Variations via Introspective Transformation Network
Authors: Yunhan Zhao, Ye Tian, Charless Fowlkes, Wei Shen, Alan Yuille
Date: 2018-05-16 00:00:00
Tasks: Data Augmentation, Few-Shot Learning

Training deep networks that generalize to a wide range of variations in test data is essential to building accurate and robust image classifiers. One standard strategy is to apply data augmentation to synthetically enlarge the training set. However, data augmentation is essentially a brute-force method which generates uniform samples from some pre-defined set of transformations. In this paper, we propose a principled approach to train networks with significantly improved resistance to large variations between training and testing data. This is achieved by embedding a learnable transformation module into the introspective network, which is a convolutional neural network (CNN) classifier empowered with generative capabilities. Our approach alternates between synthesizing pseudo-negative samples and transformed positive examples based on the current model, and optimizing model predictions on these synthesized samples. Experimental results verify that our approach significantly improves the ability of deep networks to resist large variations between training and testing data and achieves classification accuracy improvements on several benchmark datasets, including MNIST, affNIST, SVHN, CIFAR-10 and miniImageNet.
Regularized Finite Dimensional Kernel Sobolev Discrepancy
Authors: Youssef Mroueh
Date: 2018-05-16 00:00:00

We show in this note that the Sobolev Discrepancy introduced in Mroueh et al
in the context of generative adversarial networks, is actually the weighted
negative Sobolev norm $||.||_{\dot{H}^{-1}(\nu_q)}$, that is known to linearize
the Wasserstein $W_2$ distance and plays a fundamental role in the dynamic
formulation of optimal transport of Benamou and Brenier. Given a Kernel with
finite dimensional feature map we show that the Sobolev discrepancy can be
approximated from finite samples. Assuming this discrepancy is finite, the
error depends on the approximation error in the function space induced by the
finite dimensional feature space kernel and on a statistical error due to the
finite sample approximation.
Regularization Learning Networks: Deep Learning for Tabular Datasets
Authors: Ira Shavitt, Eran Segal
Date: 2018-05-16 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: counterfactual, Deep Learning

Despite their impressive performance, Deep Neural Networks (DNNs) typically
underperform Gradient Boosting Trees (GBTs) on many tabular-dataset learning
tasks. We propose that applying a different regularization coefficient to each
weight might boost the performance of DNNs by allowing them to make more use of
the more relevant inputs. However, this will lead to an intractable number of
hyperparameters. Here, we introduce Regularization Learning Networks (RLNs),
which overcome this challenge by introducing an efficient hyperparameter tuning
scheme which minimizes a new Counterfactual Loss. Our results show that RLNs
significantly improve DNNs on tabular datasets, and achieve comparable results
to GBTs, with the best performance achieved with an ensemble that combines GBTs
and RLNs. RLNs produce extremely sparse networks, eliminating up to 99.8% of
the network edges and 82% of the input features, thus providing more
interpretable models and reveal the importance that the network assigns to
different inputs. RLNs could efficiently learn a single network in datasets
that comprise both tabular and unstructured data, such as in the setting of
medical imaging accompanied by electronic health records. An open source
implementation of RLN can be found at
https://github.com/irashavitt/regularization_learning_networks.
Prediction Rule Reshaping
Authors: Matt Bonakdarpour, Sabyasachi Chatterjee, Rina Foygel Barber, John Lafferty
Date: 2018-05-16 00:00:00
Proceeding: ICML 2018 7
Tasks: General Classification, Prediction, regression

Two methods are proposed for high-dimensional shape-constrained regression
and classification. These methods reshape pre-trained prediction rules to
satisfy shape constraints like monotonicity and convexity. The first method can
be applied to any pre-trained prediction rule, while the second method deals
specifically with random forests. In both cases, efficient algorithms are
developed for computing the estimators, and experiments are performed to
demonstrate their performance on four datasets. We find that reshaping methods
enforce shape constraints without compromising predictive accuracy.
Stochastic Multi-armed Bandits in Constant Space
Authors: David Liau, Eric Price, Zhao Song, Ger Yang
Date: 2017-12-25 00:00:00
Tasks: Multi-Armed Bandits

We consider the stochastic bandit problem in the sublinear space setting,
where one cannot record the win-loss record for all $K$ arms. We give an
algorithm using $O(1)$ words of space with regret \[
  \sum_{i=1}^{K}\frac{1}{\Delta_i}\log \frac{\Delta_i}{\Delta}\log T \] where
$\Delta_i$ is the gap between the best arm and arm $i$ and $\Delta$ is the gap
between the best and the second-best arms. If the rewards are bounded away from
$0$ and $1$, this is within an $O(\log 1/\Delta)$ factor of the optimum regret
possible without space constraints.
Improve Uncertainty Estimation for Unknown Classes in Bayesian Neural Networks with Semi-Supervised /One Set Classification
Authors: Buu Phan
Date: 2018-05-04 00:00:00
Tasks: Autonomous Driving, General Classification, Medical Diagnosis, One-Class Classification

Although deep neural network (DNN) has achieved many state-of-the-art
results, estimating the uncertainty presented in the DNN model and the data is
a challenging task. Problems related to uncertainty such as classifying unknown
classes (class which does not appear in the training data) data as known class
with high confidence, is critically concerned in the safety domain area (e.g,
autonomous driving, medical diagnosis). In this paper, we show that applying
current Bayesian Neural Network (BNN) techniques alone does not effectively
capture the uncertainty. To tackle this problem, we introduce a simple way to
improve the BNN by using one class classification (in this paper, we use the
term "set classification" instead). We empirically show the result of our
method on an experiment which involves three datasets: MNIST, notMNIST and
FMNIST.
CASCADE: Contextual Sarcasm Detection in Online Discussion Forums
Authors: Devamanyu Hazarika, Soujanya Poria, Sruthi Gorantla, Erik Cambria, Roger Zimmermann, Rada Mihalcea
Date: 2018-05-16 00:00:00
Proceeding: COLING 2018 8
Tasks: General Classification, Sarcasm Detection, Sentence

The literature in automated sarcasm detection has mainly focused on lexical,
syntactic and semantic-level analysis of text. However, a sarcastic sentence
can be expressed with contextual presumptions, background and commonsense
knowledge. In this paper, we propose CASCADE (a ContextuAl SarCasm DEtector)
that adopts a hybrid approach of both content and context-driven modeling for
sarcasm detection in online social media discussions. For the latter, CASCADE
aims at extracting contextual information from the discourse of a discussion
thread. Also, since the sarcastic nature and form of expression can vary from
person to person, CASCADE utilizes user embeddings that encode stylometric and
personality features of the users. When used along with content-based feature
extractors such as Convolutional Neural Networks (CNNs), we see a significant
boost in the classification performance on a large Reddit corpus.
Deep Segmentation and Registration in X-Ray Angiography Video
Authors: Athanasios Vlontzos, Krystian Mikolajczyk
Date: 2018-05-16 00:00:00
Tasks: Optical Flow Estimation, Position, Segmentation, Semantic Segmentation

In interventional radiology, short video sequences of vein structure in
motion are captured in order to help medical personnel identify vascular issues
or plan intervention. Semantic segmentation can greatly improve the usefulness
of these videos by indicating exact position of vessels and instruments, thus
reducing the ambiguity. We propose a real-time segmentation method for these
tasks, based on U-Net network trained in a Siamese architecture from
automatically generated annotations. We make use of noisy low level binary
segmentation and optical flow to generate multi class annotations that are
successively improved in a multistage segmentation approach. We significantly
improve the performance of a state of the art U-Net at the processing speeds of
90fps.
When Regression Meets Manifold Learning for Object Recognition and Pose Estimation
Authors: Mai Bui, Sergey Zakharov, Shadi Albarqouni, Slobodan Ilic, Nassir Navab
Date: 2018-05-16 00:00:00
Tasks: Multi-Task Learning, Object Recognition, Pose Estimation, Pose Retrieval, regression, Retrieval, Triplet

In this work, we propose a method for object recognition and pose estimation
from depth images using convolutional neural networks. Previous methods
addressing this problem rely on manifold learning to learn low dimensional
viewpoint descriptors and employ them in a nearest neighbor search on an
estimated descriptor space. In comparison we create an efficient multi-task
learning framework combining manifold descriptor learning and pose regression.
By combining the strengths of manifold learning using triplet loss and pose
regression, we could either estimate the pose directly reducing the complexity
compared to NN search, or use learned descriptor for the NN descriptor
matching. By in depth experimental evaluation of the novel loss function we
observed that the view descriptors learned by the network are much more
discriminative resulting in almost 30% increase regarding relative pose
accuracy compared to related works. On the other hand, regarding directly
regressed poses we obtained important improvement compared to simple pose
regression. By leveraging the advantages of both manifold learning and
regression tasks, we are able to improve the current state-of-the-art for
object recognition and pose retrieval that we demonstrate through in depth
experimental evaluation.
Neural Multi-scale Image Compression
Authors: Ken Nakanishi, Shin-ichi Maeda, Takeru Miyato, Daisuke Okanohara
Date: 2018-05-16 00:00:00
Tasks: CPU, GPU, Image Compression

This study presents a new lossy image compression method that utilizes the
multi-scale features of natural images. Our model consists of two networks:
multi-scale lossy autoencoder and parallel multi-scale lossless coder. The
multi-scale lossy autoencoder extracts the multi-scale image features to
quantized variables and the parallel multi-scale lossless coder enables rapid
and accurate lossless coding of the quantized variables via encoding/decoding
the variables in parallel. Our proposed model achieves comparable performance
to the state-of-the-art model on Kodak and RAISE-1k dataset images, and it
encodes a PNG image of size $768 \times 512$ in 70 ms with a single GPU and a
single CPU process and decodes it into a high-fidelity image in approximately
200 ms.
Composing Finite State Transducers on GPUs
Authors: Arturo Argueta, David Chiang
Date: 2018-05-16 00:00:00
Proceeding: ACL 2018 7
Tasks: CPU, GPU, Part-Of-Speech Tagging, speech-recognition, Speech Recognition

Weighted finite-state transducers (FSTs) are frequently used in language
processing to handle tasks such as part-of-speech tagging and speech
recognition. There has been previous work using multiple CPU cores to
accelerate finite state algorithms, but limited attention has been given to
parallel graphics processing unit (GPU) implementations. In this paper, we
introduce the first (to our knowledge) GPU implementation of the FST
composition operation, and we also discuss the optimizations used to achieve
the best performance on this architecture. We show that our approach obtains
speedups of up to 6x over our serial implementation and 4.5x over OpenFST.
Nonlinear Metric Learning through Geodesic Interpolation within Lie Groups
Authors: Zhewei Wang, Bibo Shi, Charles D. Smith, Jundong Liu
Date: 2018-05-12 00:00:00
Tasks: General Classification, Metric Learning

In this paper, we propose a nonlinear distance metric learning scheme based
on the fusion of component linear metrics. Instead of merging displacements at
each data point, our model calculates the velocities induced by the component
transformations, via a geodesic interpolation on a Lie transfor- mation group.
Such velocities are later summed up to produce a global transformation that is
guaranteed to be diffeomorphic. Consequently, pair-wise distances computed this
way conform to a smooth and spatially varying metric, which can greatly benefit
k-NN classification. Experiments on synthetic and real datasets demonstrate the
effectiveness of our model.
#phramacovigilance - Exploring Deep Learning Techniques for Identifying Mentions of Medication Intake from Twitter
Authors: Debanjan Mahata, Jasper Friedrichs, Hitkul, Rajiv Ratn Shah
Date: 2018-05-16 00:00:00
Tasks: Pharmacovigilance

Mining social media messages for health and drug related information has
received significant interest in pharmacovigilance research. Social media sites
(e.g., Twitter), have been used for monitoring drug abuse, adverse reactions of
drug usage and analyzing expression of sentiments related to drugs. Most of
these studies are based on aggregated results from a large population rather
than specific sets of individuals. In order to conduct studies at an individual
level or specific cohorts, identifying posts mentioning intake of medicine by
the user is necessary. Towards this objective, we train different deep neural
network classification models on a publicly available annotated dataset and
study their performances on identifying mentions of personal intake of medicine
in tweets. We also design and train a new architecture of a stacked ensemble of
shallow convolutional neural network (CNN) ensembles. We use random search for
tuning the hyperparameters of the models and share the details of the values
taken by the hyperparameters for the best learnt model in different deep neural
network architectures. Our system produces state-of-the-art results, with a
micro- averaged F-score of 0.693.
Progress & Compress: A scalable framework for continual learning
Authors: Jonathan Schwarz, Jelena Luketina, Wojciech M. Czarnecki, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, Raia Hadsell
Date: 2018-05-16 00:00:00
Proceeding: ICML 2018 7
Tasks: Active Learning, Atari Games, Continual Learning, Reinforcement Learning

We introduce a conceptually simple and scalable framework for continual
learning domains where tasks are learned sequentially. Our method is constant
in the number of parameters and is designed to preserve performance on
previously encountered tasks while accelerating learning progress on subsequent
problems. This is achieved by training a network with two components: A
knowledge base, capable of solving previously encountered problems, which is
connected to an active column that is employed to efficiently learn the current
task. After learning a new task, the active column is distilled into the
knowledge base, taking care to protect any previously acquired skills. This
cycle of active learning (progression) followed by consolidation (compression)
requires no architecture growth, no access to or storing of previous data or
tasks, and no task-specific parameters. We demonstrate the progress & compress
approach on sequential classification of handwritten alphabets as well as two
reinforcement learning domains: Atari games and 3D maze navigation.
Towards Complex Artificial Life
Authors: Lance R. Williams
Date: 2018-05-16 00:00:00
Tasks: Artificial Life

An object-oriented combinator chemistry was used to construct an artificial
organism with a system architecture possessing characteristics necessary for
organisms to evolve into more complex forms. This architecture supports
modularity by providing a mechanism for the construction of executable modules
called $methods$ that can be duplicated and specialized to increase complexity.
At the same time, its support for concurrency provides the flexibility in
execution order necessary for redundancy, degeneracy and parallelism to
mitigate increased replication costs. The organism is a moving,
self-replicating, spatially distributed assembly of elemental combinators
called a $roving \: pile.$ The pile hosts an asynchronous message passing
computation implemented by parallel subprocesses encoded by genes distributed
through out the pile like the plasmids of a bacterial cell.