pwc-archive/papers-with-abstracts huggingface dataset:
Multimodal Generative Models for Scalable Weakly-Supervised Learning
Authors: Mike Wu, Noah Goodman
Date: 2018-02-14 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Colorization, Edge Detection, Machine Translation, Translation, Weakly-supervised Learning

Multiple modalities often co-occur when describing natural phenomena.
Learning a joint representation of these modalities should yield deeper and
more useful representations. Previous generative approaches to multi-modal
input either do not learn a joint distribution or require additional
computation to handle missing data. Here, we introduce a multimodal variational
autoencoder (MVAE) that uses a product-of-experts inference network and a
sub-sampled training paradigm to solve the multi-modal inference problem.
Notably, our model shares parameters to efficiently learn under any combination
of missing modalities. We apply the MVAE on four datasets and match
state-of-the-art performance using many fewer parameters. In addition, we show
that the MVAE is directly applicable to weakly-supervised learning, and is
robust to incomplete supervision. We then consider two case studies, one of
learning image transformations---edge detection, colorization,
segmentation---as a set of modalities, followed by one of machine translation
between two languages. We find appealing results across this range of tasks.
Semantic Adversarial Deep Learning
Authors: Tommaso Dreossi, Somesh Jha, Sanjit A. Seshia
Date: 2018-04-19 00:00:00
Tasks: Deep Learning, Malware Detection, Self-Driving Cars

Fueled by massive amounts of data, models produced by machine-learning (ML)
algorithms, especially deep neural networks, are being used in diverse domains
where trustworthiness is a concern, including automotive systems, finance,
health care, natural language processing, and malware detection. Of particular
concern is the use of ML algorithms in cyber-physical systems (CPS), such as
self-driving cars and aviation, where an adversary can cause serious
consequences. However, existing approaches to generating adversarial examples
and devising robust ML algorithms mostly ignore the semantics and context of
the overall system containing the ML component. For example, in an autonomous
vehicle using deep learning for perception, not every adversarial example for
the neural network might lead to a harmful consequence. Moreover, one may want
to prioritize the search for adversarial examples towards those that
significantly modify the desired semantics of the overall system. Along the
same lines, existing algorithms for constructing robust ML algorithms ignore
the specification of the overall system. In this paper, we argue that the
semantics and specification of the overall system has a crucial role to play in
this line of research. We present preliminary research results that support
this claim.
GumBolt: Extending Gumbel trick to Boltzmann priors
Authors: Amir H. Khoshaman, Mohammad H. Amin
Date: 2018-05-18 00:00:00
Proceeding: NeurIPS 2018 12

Boltzmann machines (BMs) are appealing candidates for powerful priors in
variational autoencoders (VAEs), as they are capable of capturing nontrivial
and multi-modal distributions over discrete variables. However,
non-differentiability of the discrete units prohibits using the
reparameterization trick, essential for low-noise back propagation. The Gumbel
trick resolves this problem in a consistent way by relaxing the variables and
distributions, but it is incompatible with BM priors. Here, we propose the
GumBolt, a model that extends the Gumbel trick to BM priors in VAEs. GumBolt is
significantly simpler than the recently proposed methods with BM prior and
outperforms them by a considerable margin. It achieves state-of-the-art
performance on permutation invariant MNIST and OMNIGLOT datasets in the scope
of models with only discrete latent variables. Moreover, the performance can be
further improved by allowing multi-sampled (importance-weighted) estimation of
log-likelihood in training, which was not possible with previous models.
A Generalized Circuit for the Hamiltonian Dynamics Through the Truncated Series
Authors: Ammar Daskin, Sabre Kais
Date: 2018-01-29 00:00:00

In this paper, we present a method for the Hamiltonian simulation in the
context of eigenvalue estimation problems which improves earlier results
dealing with Hamiltonian simulation through the truncated Taylor series. In
particular, we present a fixed-quantum circuit design for the simulation of the
Hamiltonian dynamics, $H(t)$, through the truncated Taylor series method
described by Berry et al. \cite{berry2015simulating}. The circuit is general
and can be used to simulate any given matrix in the phase estimation algorithm
by only changing the angle values of the quantum gates implementing the time
variable $t$ in the series. The circuit complexity depends on the number of
summation terms composing the Hamiltonian and requires $O(Ln)$ number of
quantum gates for the simulation of a molecular Hamiltonian. Here, $n$ is the
number of states of a spin orbital, and $L$ is the number of terms in the
molecular Hamiltonian and generally bounded by $O(n^4)$. We also discuss how to
use the circuit in adaptive processes and eigenvalue related problems along
with a slight modified version of the iterative phase estimation algorithm. In
addition, a simple divide and conquer method is presented for mapping a matrix
which are not given as sums of unitary matrices into the circuit. The
complexity of the circuit is directly related to the structure of the matrix
and can be bounded by $O(poly(n))$ for a matrix with $poly(n)-$sparsity.
Accurate Kernel Learning for Linear Gaussian Markov Processes using a Scalable Likelihood Computation
Authors: Stijn de Waele
Date: 2018-05-18 00:00:00

We report an exact likelihood computation for Linear Gaussian Markov
processes that is more scalable than existing algorithms for complex models and
sparsely sampled signals. Better scaling is achieved through elimination of
repeated computations in the Kalman likelihood, and by using the diagonalized
form of the state transition equation. Using this efficient computation, we
study the accuracy of kernel learning using maximum likelihood and the
posterior mean in a simulation experiment. The posterior mean with a reference
prior is more accurate for complex models and sparse sampling. Because of its
lower computation load, the maximum likelihood estimator is an attractive
option for more densely sampled signals and lower order models. We confirm
estimator behavior in experimental data through their application to speleothem
data.
Cancer Research UK Drug Discovery Process Mining
Authors: Haochao Huang
Date: 2018-05-18 00:00:00
Tasks: Drug Discovery, Management

Background. The Drug Discovery Unit (DDU) of Cancer Research UK (CRUK) is
using the software Dotmatics for storage and analysis of scientific data during
drug discovery process. Whilst the data include event logs, time stamps,
activities, and user information are mostly sitting in the database without
fully utilising their potential value. Aims. This dissertation aims at
extracting knowledge from event logs data which recorded during drug discovery
process, to capture the operational business process of the DDU of Cancer
Research UK (CRUK) as it was being executed. It provides the evaluations and
methodologies of drawing the process mining panoramic models for the drug
discovery process. Thus by enabling the DDU to maximise its efficiency in
reviewing its resources and works allocations, patients will benefit from more
new treatments faster. Conclusion. Management of organisations can be benefit
from the process mining methodologies. Disco is excellent for non-experts on
management purposes. ProM is great for expert on research purposes. However,
the process mining is not once and for all but is a regular operation
management process. Indeed, event logs needs to be understand more on the
target organisational behaviours and organisational business process. The
researchers have to be aware that event logs data are the most important and
priority elements in process mining.
Improved Sentence Modeling using Suffix Bidirectional LSTM
Authors: Siddhartha Brahma
Date: 2018-05-18 00:00:00
Tasks: Classification, General Classification, Natural Language Inference, Sentence, Sentiment Analysis, Sentiment Classification, text-classification, Text Classification

Recurrent neural networks have become ubiquitous in computing representations
of sequential data, especially textual data in natural language processing. In
particular, Bidirectional LSTMs are at the heart of several neural models
achieving state-of-the-art performance in a wide variety of tasks in NLP.
However, BiLSTMs are known to suffer from sequential bias - the contextual
representation of a token is heavily influenced by tokens close to it in a
sentence. We propose a general and effective improvement to the BiLSTM model
which encodes each suffix and prefix of a sequence of tokens in both forward
and reverse directions. We call our model Suffix Bidirectional LSTM or
SuBiLSTM. This introduces an alternate bias that favors long range
dependencies. We apply SuBiLSTMs to several tasks that require sentence
modeling. We demonstrate that using SuBiLSTM instead of a BiLSTM in existing
models leads to improvements in performance in learning general sentence
representations, text classification, textual entailment and paraphrase
detection. Using SuBiLSTM we achieve new state-of-the-art results for
fine-grained sentiment classification and question classification.
Scanner: Efficient Video Analysis at Scale
Authors: Alex Poms, Will Crichton, Pat Hanrahan, Kayvon Fatahalian
Date: 2018-05-18 00:00:00

A growing number of visual computing applications depend on the analysis of
large video collections. The challenge is that scaling applications to operate
on these datasets requires efficient systems for pixel data access and parallel
processing across large numbers of machines. Few programmers have the
capability to operate efficiently at these scales, limiting the field's ability
to explore new applications that leverage big video data. In response, we have
created Scanner, a system for productive and efficient video analysis at scale.
Scanner organizes video collections as tables in a data store optimized for
sampling frames from compressed video, and executes pixel processing
computations, expressed as dataflow graphs, on these frames. Scanner schedules
video analysis applications expressed using these abstractions onto
heterogeneous throughput computing hardware, such as multi-core CPUs, GPUs, and
media processing ASICs, for high-throughput pixel processing. We demonstrate
the productivity of Scanner by authoring a variety of video processing
applications including the synthesis of stereo VR video streams from
multi-camera rigs, markerless 3D human pose reconstruction from video, and
data-mining big video datasets such as hundreds of feature-length films or over
70,000 hours of TV news. These applications achieve near-expert performance on
a single machine and scale efficiently to hundreds of machines, enabling
formerly long-running big video data analysis tasks to be carried out in
minutes to hours.
Reconstruction of training samples from loss functions
Authors: Akiyoshi Sannai
Date: 2018-05-18 00:00:00

This paper presents a new mathematical framework to analyze the loss
functions of deep neural networks with ReLU functions. Furthermore, as as
application of this theory, we prove that the loss functions can reconstruct
the inputs of the training samples up to scalar multiplication (as vectors) and
can provide the number of layers and nodes of the deep neural network. Namely,
if we have all input and output of a loss function (or equivalently all
possible learning process), for all input of each training sample $x_i \in
\mathbb{R}^n$, we can obtain vectors $x'_i\in \mathbb{R}^n$ satisfying
$x_i=c_ix'_i$ for some $c_i \neq 0$. To prove theorem, we introduce the notion
of virtual polynomials, which are polynomials written as the output of a node
in a deep neural network. Using virtual polynomials, we find an algebraic
structure for the loss surfaces, called semi-algebraic sets. We analyze these
loss surfaces from the algebro-geometric point of view. Factorization of
polynomials is one of the most standard ideas in algebra. Hence, we express the
factorization of the virtual polynomials in terms of their active paths. This
framework can be applied to the leakage problem in the training of deep neural
networks. The main theorem in this paper indicates that there are many risks
associated with the training of deep neural networks. For example, if we have N
(the dimension of weight space) + 1 nonsmooth points on the loss surface, which
are sufficiently close to each other, we can obtain the input of training
sample up to scalar multiplication. We also point out that the structures of
the loss surfaces depend on the shape of the deep neural network and not on the
training samples.
Positive and Unlabeled Learning through Negative Selection and Imbalance-aware Classification
Authors: Marco Frasca, Nicolò Cesa-Bianchi
Date: 2018-05-18 00:00:00
Tasks: Active Learning, General Classification, Prediction, Protein Function Prediction

Motivated by applications in protein function prediction, we consider a
challenging supervised classification setting in which positive labels are
scarce and there are no explicit negative labels. The learning algorithm must
thus select which unlabeled examples to use as negative training points,
possibly ending up with an unbalanced learning problem. We address these issues
by proposing an algorithm that combines active learning (for selecting negative
examples) with imbalance-aware learning (for mitigating the label imbalance).
In our experiments we observe that these two techniques operate
synergistically, outperforming state-of-the-art methods on standard protein
function prediction benchmarks.
Human Motion Modeling using DVGANs
Authors: Xiao Lin, Mohamed R. Amer
Date: 2018-04-27 00:00:00
Tasks: Motion Generation, Translation

We present a novel generative model for human motion modeling using
Generative Adversarial Networks (GANs). We formulate the GAN discriminator
using dense validation at each time-scale and perturb the discriminator input
to make it translation invariant. Our model is capable of motion generation and
completion. We show through our evaluations the resiliency to noise,
generalization over actions, and generation of long diverse sequences. We
evaluate our approach on Human 3.6M and CMU motion capture datasets using
inception scores.
Mixup-Based Acoustic Scene Classification Using Multi-Channel Convolutional Neural Network
Authors: Kele Xu, Dawei Feng, Haibo Mi, Boqing Zhu, Dezhi Wang, Lilun Zhang, Hengxing Cai, Shuwen Liu
Date: 2018-05-18 00:00:00
Tasks: Acoustic Scene Classification, Classification, General Classification, Scene Classification

Audio scene classification, the problem of predicting class labels of audio
scenes, has drawn lots of attention during the last several years. However, it
remains challenging and falls short of accuracy and efficiency. Recently,
Convolutional Neural Network (CNN)-based methods have achieved better
performance with comparison to the traditional methods. Nevertheless,
conventional single channel CNN may fail to consider the fact that additional
cues may be embedded in the multi-channel recordings. In this paper, we explore
the use of Multi-channel CNN for the classification task, which aims to extract
features from different channels in an end-to-end manner. We conduct the
evaluation compared with the conventional CNN and traditional Gaussian Mixture
Model-based methods. Moreover, to improve the classification accuracy further,
this paper explores the using of mixup method. In brief, mixup trains the
neural network on linear combinations of pairs of the representation of audio
scene examples and their labels. By employing the mixup approach for data
argumentation, the novel model can provide higher prediction accuracy and
robustness in contrast with previous models, while the generalization error can
also be reduced on the evaluation data.
Efficient Exploration of Gradient Space for Online Learning to Rank
Authors: Huazheng Wang, Ramsey Langley, Sonwoo Kim, Eric McCord-Snook, Hongning Wang
Date: 2018-05-18 00:00:00
Tasks: Efficient Exploration, Learning-To-Rank

Online learning to rank (OL2R) optimizes the utility of returned search
results based on implicit feedback gathered directly from users. To improve the
estimates, OL2R algorithms examine one or more exploratory gradient directions
and update the current ranker if a proposed one is preferred by users via an
interleaved test. In this paper, we accelerate the online learning process by
efficient exploration in the gradient space. Our algorithm, named as Null Space
Gradient Descent, reduces the exploration space to only the \emph{null space}
of recent poorly performing gradients. This prevents the algorithm from
repeatedly exploring directions that have been discouraged by the most recent
interactions with users. To improve sensitivity of the resulting interleaved
test, we selectively construct candidate rankers to maximize the chance that
they can be differentiated by candidate ranking documents in the current query;
and we use historically difficult queries to identify the best ranker when tie
occurs in comparing the rankers. Extensive experimental comparisons with the
state-of-the-art OL2R algorithms on several public benchmarks confirmed the
effectiveness of our proposal algorithm, especially in its fast learning
convergence and promising ranking quality at an early stage.
Blended Conditional Gradients: the unconditioning of conditional gradients
Authors: Gábor Braun, Sebastian Pokutta, Dan Tu, Stephen Wright
Date: 2018-05-18 00:00:00

We present a blended conditional gradient approach for minimizing a smooth convex function over a polytope P, combining the Frank--Wolfe algorithm (also called conditional gradient) with gradient-based steps, different from away steps and pairwise steps, but still achieving linear convergence for strongly convex functions, along with good practical performance. Our approach retains all favorable properties of conditional gradient algorithms, notably avoidance of projections onto P and maintenance of iterates as sparse convex combinations of a limited number of extreme points of P. The algorithm is lazy, making use of inexpensive inexact solutions of the linear programming subproblem that characterizes the conditional gradient approach. It decreases measures of optimality (primal and dual gaps) rapidly, both in the number of iterations and in wall-clock time, outperforming even the lazy conditional gradient algorithms of [arXiv:1410.8816]. We also present a streamlined version of the algorithm for the probability simplex.
Unsupervised Semantic Frame Induction using Triclustering
Authors: Dmitry Ustalov, Alexander Panchenko, Andrei Kutuzov, Chris Biemann, Simone Paolo Ponzetto
Date: 2018-05-12 00:00:00
Proceeding: ACL 2018 7
Tasks: Clustering

We use dependency triples automatically extracted from a Web-scale corpus to
perform unsupervised semantic frame induction. We cast the frame induction
problem as a triclustering problem that is a generalization of clustering for
triadic data. Our replicable benchmarks demonstrate that the proposed
graph-based approach, Triframes, shows state-of-the art results on this task on
a FrameNet-derived dataset and performing on par with competitive methods on a
verb class clustering task.
Probing hidden spin order with interpretable machine learning
Authors: Jonas Greitemann, Ke Liu, Lode Pollet
Date: 2018-04-23 00:00:00
Tasks: BIG-bench Machine Learning, Interpretable Machine Learning

The search of unconventional magnetic and nonmagnetic states is a major topic
in the study of frustrated magnetism. Canonical examples of those states
include various spin liquids and spin nematics. However, discerning their
existence and the correct characterization is usually challenging. Here we
introduce a machine-learning protocol that can identify general nematic order
and their order parameter from seemingly featureless spin configurations, thus
providing comprehensive insight on the presence or absence of hidden orders. We
demonstrate the capabilities of our method by extracting the analytical form of
nematic order parameter tensors up to rank 6. This may prove useful in the
search for novel spin states and for ruling out spurious spin liquid
candidates.
Multitaper Spectral Estimation HDP-HMMs for EEG Sleep Inference
Authors: Leon Chlon, Andrew Song, Sandya Subramanian, Hugo Soulat, John Tauber, Demba Ba, Michael Prerau
Date: 2018-05-18 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Time Series, Time Series Analysis

Electroencephalographic (EEG) monitoring of neural activity is widely used
for sleep disorder diagnostics and research. The standard of care is to
manually classify 30-second epochs of EEG time-domain traces into 5 discrete
sleep stages. Unfortunately, this scoring process is subjective and
time-consuming, and the defined stages do not capture the heterogeneous
landscape of healthy and clinical neural dynamics. This motivates the search
for a data-driven and principled way to identify the number and composition of
salient, reoccurring brain states present during sleep. To this end, we propose
a Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM), combined with
wide-sense stationary (WSS) time series spectral estimation to construct a
generative model for personalized subject sleep states. In addition, we employ
multitaper spectral estimation to further reduce the large variance of the
spectral estimates inherent to finite-length EEG measurements. By applying our
method to both simulated and human sleep data, we arrive at three main results:
1) a Bayesian nonparametric automated algorithm that recovers general temporal
dynamics of sleep, 2) identification of subject-specific "microstates" within
canonical sleep stages, and 3) discovery of stage-dependent sub-oscillations
with shared spectral signatures across subjects.
LMNet: Real-time Multiclass Object Detection on CPU using 3D LiDAR
Authors: Kazuki Minemura, Hengfui Liau, Abraham Monrroy, Shinpei Kato
Date: 2018-05-13 00:00:00
Tasks: Autonomous Driving, CPU, object-detection, Object Detection

This paper describes an optimized single-stage deep convolutional neural
network to detect objects in urban environments, using nothing more than point
cloud data. This feature enables our method to work regardless the time of the
day and the lighting conditions.The proposed network structure employs dilated
convolutions to gradually increase the perceptive field as depth increases,
this helps to reduce the computation time by about 30%. The network input
consists of five perspective representations of the unorganized point cloud
data. The network outputs an objectness map and the bounding box offset values
for each point. Our experiments showed that using reflection, range, and the
position on each of the three axes helped to improve the location and
orientation of the output bounding box. We carried out quantitative evaluations
with the help of the KITTI dataset evaluation server. It achieved the fastest
processing speed among the other contenders, making it suitable for real-time
applications. We implemented and tested it on a real vehicle with a Velodyne
HDL-64 mounted on top of it. We achieved execution times as fast as 50 FPS
using desktop GPUs, and up to 10 FPS on a single Intel Core i5 CPU. The deploy
implementation is open-sourced and it can be found as a feature branch inside
the autonomous driving framework Autoware. Code is available at:
https://github.com/CPFL/Autoware/tree/feature/cnn_lidar_detection