pwc-archive/papers-with-abstracts huggingface dataset:
A Deep, Information-theoretic Framework for Robust Biometric Recognition
Authors: Renjie Xie, Yanzhi Chen, Yan Wo, Qiao Wang
Date: 2019-02-23 00:00:00
Tasks: valid

Deep neural networks (DNN) have been a de facto standard for nowadays
biometric recognition solutions. A serious, but still overlooked problem in
these DNN-based recognition systems is their vulnerability against adversarial
attacks. Adversarial attacks can easily cause the output of a DNN system to
greatly distort with only tiny changes in its input. Such distortions can
potentially lead to an unexpected match between a valid biometric and a
synthetic one constructed by a strategic attacker, raising security issue. In
this work, we show how this issue can be resolved by learning robust biometric
features through a deep, information-theoretic framework, which builds upon the
recent deep variational information bottleneck method but is carefully adapted
to biometric recognition tasks. Empirical evaluation demonstrates that our
method not only offers stronger robustness against adversarial attacks but also
provides better recognition performance over state-of-the-art approaches.
Quantum Learning Boolean Linear Functions w.r.t. Product Distributions
Authors: Matthias C. Caro
Date: 2019-02-23 00:00:00

The problem of learning Boolean linear functions from quantum examples w.r.t. the uniform distribution can be solved on a quantum computer using the Bernstein-Vazirani algorithm. A similar strategy can be applied in the case of noisy quantum training data, as was observed in arXiv:1702.08255v2 [quant-ph]. However, extensions of these learning algorithms beyond the uniform distribution have not yet been studied. We employ the biased quantum Fourier transform introduced in arXiv:1802.05690v2 [quant-ph] to develop efficient quantum algorithms for learning Boolean linear functions on $n$ bits from quantum examples w.r.t. a biased product distribution. Our first procedure is applicable to any (except full) bias and requires $\mathcal{O}(\ln (n))$ quantum examples. The number of quantum examples used by our second algorithm is independent of $n$, but the strategy is applicable only for small bias. Moreover, we show that the second procedure is stable w.r.t. noisy training data and w.r.t. faulty quantum gates. This also enables us to solve a version of the learning problem in which the underlying distribution is not known in advance. Finally, we prove lower bounds on the classical and quantum sample complexities of the learning problem. Whereas classically, $\Omega (n)$ examples are necessary independently of the bias, we are able to establish a quantum sample complexity lower bound of $\Omega (\ln (n))$ only under an assumption of large bias. Nevertheless, this allows for a discussion of the performance of our suggested learning algorithms w.r.t. sample complexity. With our analysis we contribute to a more quantitative understanding of the power and limitations of quantum training data for learning classical functions.
Behavioral Petri Net Mining and Automated Analysis for Human-Computer Interaction Recommendations in Multi-Application Environments
Authors: Julian Theis, Houshang Darabi
Date: 2019-02-23 00:00:00

Process Mining is a famous technique which is frequently applied to Software
Development Processes, while being neglected in Human-Computer Interaction
(HCI) recommendation applications. Organizations usually train employees to
interact with required IT systems. Often, employees, or users in general,
develop their own strategies for solving repetitive tasks and processes.
However, organizations find it hard to detect whether employees interact
efficiently with IT systems or not. Hence, we have developed a method which
detects inefficient behavior assuming that at least one optimal HCI strategy is
known. This method provides recommendations to gradually adapt users' behavior
towards the optimal way of interaction considering satisfaction of users. Based
on users' behavior logs tracked by a Java application suitable for
multi-application and multi-instance environments, we demonstrate the
applicability for a specific task in a common Windows environment utilizing
realistic simulated behaviors of users.
Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach
Authors: Minyoung Kim, Pritish Sahu, Behnam Gholami, Vladimir Pavlovic
Date: 2019-02-23 00:00:00
Proceeding: CVPR 2019 6
Tasks: Domain Adaptation, Gaussian Processes, Unsupervised Domain Adaptation

In unsupervised domain adaptation, it is widely known that the target domain
error can be provably reduced by having a shared input representation that
makes the source and target domains indistinguishable from each other. Very
recently it has been studied that not just matching the marginal input
distributions, but the alignment of output (class) distributions is also
critical. The latter can be achieved by minimizing the maximum discrepancy of
predictors (classifiers). In this paper, we adopt this principle, but propose a
more systematic and effective way to achieve hypothesis consistency via
Gaussian processes (GP). The GP allows us to define/induce a hypothesis space
of the classifiers from the posterior distribution of the latent random
functions, turning the learning into a simple large-margin posterior separation
problem, far easier to solve than previous approaches based on adversarial
minimax optimization. We formulate a learning objective that effectively pushes
the posterior to minimize the maximum discrepancy. This is further shown to be
equivalent to maximizing margins and minimizing uncertainty of the class
predictions in the target domain, a well-established principle in classical
(semi-)supervised learning. Empirical results demonstrate that our approach is
comparable or superior to the existing methods on several benchmark domain
adaptation datasets.
A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks
Authors: Hadi Salman, Greg Yang, huan zhang, Cho-Jui Hsieh, Pengchuan Zhang
Date: 2019-02-23 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: CPU

Verification of neural networks enables us to gauge their robustness against adversarial attacks. Verification algorithms fall into two categories: exact verifiers that run in exponential time and relaxed verifiers that are efficient but incomplete. In this paper, we unify all existing LP-relaxed verifiers, to the best of our knowledge, under a general convex relaxation framework. This framework works for neural networks with diverse architectures and nonlinearities and covers both primal and dual views of robustness verification. We further prove strong duality between the primal and dual problems under very mild conditions. Next, we perform large-scale experiments, amounting to more than 22 CPU-years, to obtain exact solution to the convex-relaxed problem that is optimal within our framework for ReLU networks. We find the exact solution does not significantly improve upon the gap between PGD and existing relaxed verifiers for various networks trained normally or robustly on MNIST and CIFAR datasets. Our results suggest there is an inherent barrier to tight verification for the large class of methods captured by our framework. We discuss possible causes of this barrier and potential future directions for bypassing it. Our code and trained models are available at http://github.com/Hadisalman/robust-verify-benchmark .
Online Control with Adversarial Disturbances
Authors: Naman Agarwal, Brian Bullins, Elad Hazan, Sham M. Kakade, Karan Singh
Date: 2019-02-23 00:00:00

We study the control of a linear dynamical system with adversarial
disturbances (as opposed to statistical noise). The objective we consider is
one of regret: we desire an online control procedure that can do nearly as well
as that of a procedure that has full knowledge of the disturbances in
hindsight. Our main result is an efficient algorithm that provides nearly tight
regret bounds for this problem. From a technical standpoint, this work
generalizes upon previous work in two main aspects: our model allows for
adversarial noise in the dynamics, and allows for general convex costs.
Spatio-Temporal Convolutional LSTMs for Tumor Growth Prediction by Learning 4D Longitudinal Patient Data
Authors: Ling Zhang, Le Lu, Xiaosong Wang, Robert M. Zhu, Mohammadhadi Bagheri, Ronald M. Summers, Jianhua Yao
Date: 2019-02-23 00:00:00
Tasks: Generative Adversarial Network, Image Segmentation, Left Ventricle Segmentation, Medical Image Segmentation, Semantic Segmentation

Prognostic tumor growth modeling via volumetric medical imaging observations can potentially lead to better outcomes of tumor treatment and surgical planning. Recent advances of convolutional networks have demonstrated higher accuracy than traditional mathematical models in predicting future tumor volumes. This indicates that deep learning-based techniques may have great potentials on addressing such problem. However, current 2D patch-based modeling approaches cannot make full use of the spatio-temporal imaging context of the tumor's longitudinal 4D (3D + time) data. Moreover, they are incapable to predict clinically-relevant tumor properties, other than volumes. In this paper, we exploit to formulate the tumor growth process through convolutional Long Short-Term Memory (ConvLSTM) that extract tumor's static imaging appearances and capture its temporal dynamic changes within a single network. We extend ConvLSTM into the spatio-temporal domain (ST-ConvLSTM) by jointly learning the inter-slice 3D contexts and the longitudinal or temporal dynamics from multiple patient studies. Our approach can incorporate other non-imaging patient information in an end-to-end trainable manner. Experiments are conducted on the largest 4D longitudinal tumor dataset of 33 patients to date. Results validate that the ST-ConvLSTM produces a Dice score of 83.2%+-5.1% and a RVD of 11.2%+-10.8%, both significantly outperforming (p<0.05) other compared methods of linear model, ConvLSTM, and generative adversarial network (GAN) under the metric of predicting future tumor volumes. Additionally, our new method enables the prediction of both cell density and CT intensity numbers. Last, we demonstrate the generalizability of ST-ConvLSTM by employing it in 4D medical image segmentation task, which achieves an averaged Dice score of 86.3+-1.2% for left-ventricle segmentation in 4D ultrasound with 3 seconds per patient.
GANSynth: Adversarial Neural Audio Synthesis
Authors: Jesse Engel, Kumar Krishna Agrawal, Shuo Chen, Ishaan Gulrajani, Chris Donahue, Adam Roberts
Date: 2019-02-23 00:00:00
Proceeding: ICLR 2019 5
Tasks: Audio Generation, Audio Synthesis

Efficient audio synthesis is an inherently difficult machine learning task,
as human perception is sensitive to both global structure and fine-scale
waveform coherence. Autoregressive models, such as WaveNet, model local
structure at the expense of global latent structure and slow iterative
sampling, while Generative Adversarial Networks (GANs), have global latent
conditioning and efficient parallel sampling, but struggle to generate
locally-coherent audio waveforms. Herein, we demonstrate that GANs can in fact
generate high-fidelity and locally-coherent audio by modeling log magnitudes
and instantaneous frequencies with sufficient frequency resolution in the
spectral domain. Through extensive empirical investigations on the NSynth
dataset, we demonstrate that GANs are able to outperform strong WaveNet
baselines on automated and human evaluation metrics, and efficiently generate
audio several orders of magnitude faster than their autoregressive
counterparts.
Distributionally Robust Reinforcement Learning
Authors: Elena Smirnova, Elvis Dohmatob, Jérémie Mary
Date: 2019-02-23 00:00:00
Tasks: continuous-control, Continuous Control, Q-Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Real-world applications require RL algorithms to act safely. During learning process, it is likely that the agent executes sub-optimal actions that may lead to unsafe/poor states of the system. Exploration is particularly brittle in high-dimensional state/action space due to increased number of low-performing actions. In this work, we consider risk-averse exploration in approximate RL setting. To ensure safety during learning, we propose the distributionally robust policy iteration scheme that provides lower bound guarantee on state-values. Our approach induces a dynamic level of risk to prevent poor decisions and yet preserves the convergence to the optimal policy. Our formulation results in a efficient algorithm that accounts for a simple re-weighting of policy actions in the standard policy iteration scheme. We extend our approach to continuous state/action space and present a practical algorithm, distributionally robust soft actor-critic, that implements a different exploration strategy: it acts conservatively at short-term and it explores optimistically in a long-run. We provide promising experimental results on continuous control tasks.
Pragmatic classification of movement primitives for stroke rehabilitation
Authors: Avinash Parnandi, Jasim Uddin, Dawn M. Nilsen, Heidi Schambra
Date: 2019-02-22 00:00:00
Tasks: Activity Recognition, Classification, General Classification

Rehabilitation training is the primary intervention to improve motor recovery
after stroke, but a tool to measure functional training does not currently
exist. To bridge this gap, we previously developed an approach to classify
functional movement primitives using wearable sensors and a machine learning
(ML) algorithm. We found that this approach had encouraging classification
performance but had computational and practical limitations, such as training
time, sensor cost, and magnetic drift. Here, we sought to refine this approach
and determine the algorithm, sensor configurations, and data requirements
needed to maximize computational and practical performance.
  Motion data had been previously collected from 6 stroke patients wearing 11
inertial measurement units (IMUs) as they moved objects on a target array. To
identify optimal ML performance, we evaluated 4 algorithms that are commonly
used in activity recognition (linear discriminant analysis (LDA), na\"ive
Bayes, support vector machine, and k-nearest neighbors). We compared their
classification accuracy, computational complexity, and tuning requirements. To
identify optimal sensor configuration, we progressively sampled fewer sensors
and compared classification accuracy. To identify optimal data requirements, we
compared accuracy using data from IMUs versus accelerometers.
  We found that LDA had the highest classification accuracy (92%) of the
algorithms tested. It also was the most pragmatic, with low training and
testing times and modest tuning requirements. We found that 7 sensors on the
paretic arm and back resulted in the best accuracy. Using this array,
accelerometers had a lower accuracy (84%).
  We refined strategies to accurately and pragmatically quantify functional
movement primitives in stroke patients. We propose that this optimized
ML-sensor approach could be a means to quantify training dose after stroke.
The Generalized Complex Kernel Least-Mean-Square Algorithm
Authors: Rafael Boloix-Tortosa, Juan José Murillo-Fuentes, Sotirios A. Tsaftaris
Date: 2019-02-22 00:00:00
Tasks: regression

We propose a novel adaptive kernel based regression method for complex-valued
signals: the generalized complex-valued kernel least-mean-square (gCKLMS). We
borrow from the new results on widely linear reproducing kernel Hilbert space
(WL-RKHS) for nonlinear regression and complex-valued signals, recently
proposed by the authors. This paper shows that in the adaptive version of the
kernel regression for complex-valued signals we need to include another kernel
term, the so-called pseudo-kernel. This new solution is endowed with better
representation capabilities in complex-valued fields, since it can efficiently
decouple the learning of the real and the imaginary part. Also, we review
previous realizations of the complex KLMS algorithm and its augmented version
to prove that they can be rewritten as particular cases of the gCKLMS.
Furthermore, important conclusions on the kernels design are drawn that help to
greatly improve the convergence of the algorithms. In the experiments, we
revisit the nonlinear channel equalization problem to highlight the better
convergence of the gCKLMS compared to previous solutions. Also, the flexibility
of the proposed generalized approach is tested in a second experiment with
non-independent real and imaginary parts. The results illustrate the
significant performance improvements of the gCKLMS approach when the
complex-valued signals have different properties for the real and imaginary
parts.
Enhancing Clinical Concept Extraction with Contextual Embeddings
Authors: Yuqi Si, Jingqi Wang, Hua Xu, Kirk Roberts
Date: 2019-02-22 00:00:00
Tasks: Clinical Concept Extraction, Language Modelling, Large Language Model, Word Embeddings

Neural network-based representations ("embeddings") have dramatically advanced natural language processing (NLP) tasks, including clinical NLP tasks such as concept extraction. Recently, however, more advanced embedding methods and representations (e.g., ELMo, BERT) have further pushed the state-of-the-art in NLP, yet there are no common best practices for how to integrate these representations into clinical tasks. The purpose of this study, then, is to explore the space of possible options in utilizing these new models for clinical concept extraction, including comparing these to traditional word embedding methods (word2vec, GloVe, fastText). Both off-the-shelf open-domain embeddings and pre-trained clinical embeddings from MIMIC-III are evaluated. We explore a battery of embedding methods consisting of traditional word embeddings and contextual embeddings, and compare these on four concept extraction corpora: i2b2 2010, i2b2 2012, SemEval 2014, and SemEval 2015. We also analyze the impact of the pre-training time of a large language model like ELMo or BERT on the extraction performance. Last, we present an intuitive way to understand the semantic information encoded by contextual embeddings. Contextual embeddings pre-trained on a large clinical corpus achieves new state-of-the-art performances across all concept extraction tasks. The best-performing model outperforms all state-of-the-art methods with respective F1-measures of 90.25, 93.18 (partial), 80.74, and 81.65. We demonstrate the potential of contextual embeddings through the state-of-the-art performance these methods achieve on clinical concept extraction. Additionally, we demonstrate contextual embeddings encode valuable semantic information not accounted for in traditional word representations.
Spatial Analysis Made Easy with Linear Regression and Kernels
Authors: Philip Milton, Emanuele Giorgi, Samir Bhatt
Date: 2019-02-22 00:00:00
Tasks: regression

Kernel methods are an incredibly popular technique for extending linear
models to non-linear problems via a mapping to an implicit, high-dimensional
feature space. While kernel methods are computationally cheaper than an
explicit feature mapping, they are still subject to cubic cost on the number of
points. Given only a few thousand locations, this computational cost rapidly
outstrips the currently available computational power. This paper aims to
provide an overview of kernel methods from first-principals (with a focus on
ridge regression), before progressing to a review of random Fourier features
(RFF), a set of methods that enable the scaling of kernel methods to big
datasets. At each stage, the associated R code is provided. We begin by
illustrating how the dual representation of ridge regression relies solely on
inner products and permits the use of kernels to map the data into
high-dimensional spaces. We progress to RFFs, showing how only a few lines of
code provides a significant computational speed-up for a negligible cost to
accuracy. We provide an example of the implementation of RFFs on a simulated
spatial data set to illustrate these properties. Lastly, we summarise the main
issues with RFFs and highlight some of the advanced techniques aimed at
alleviating them.
Drug-drug interaction prediction based on co-medication patterns and graph matching
Authors: Wen-Hao Chiang, Li Shen, Lang Li, Xia Ning
Date: 2019-02-22 00:00:00
Tasks: Graph Matching, Prediction

Background: The problem of predicting whether a drug combination of arbitrary
orders is likely to induce adverse drug reactions is considered in this
manuscript. Methods: Novel kernels over drug combinations of arbitrary orders
are developed within support vector machines for the prediction. Graph matching
methods are used in the novel kernels to measure the similarities among drug
combinations, in which drug co-medication patterns are leveraged to measure
single drug similarities. Results: The experimental results on a real-world
dataset demonstrated that the new kernels achieve an area under the curve (AUC)
value 0.912 for the prediction problem. Conclusions: The new methods with drug
co-medication based single drug similarities can accurately predict whether a
drug combination is likely to induce adverse drug reactions of interest.
  Keywords: drug-drug interaction prediction; drug combination similarity;
co-medication; graph matching
Discriminative Pattern Mining for Breast Cancer Histopathology Image Classification via Fully Convolutional Autoencoder
Authors: Xingyu Li, Marko Radulovic, Ksenija Kanjer, Konstantinos N. Plataniotis
Date: 2019-02-22 00:00:00
Tasks: General Classification, image-classification, Image Classification

Accurate diagnosis of breast cancer in histopathology images is challenging due to the heterogeneity of cancer cell growth as well as of a variety of benign breast tissue proliferative lesions. In this paper, we propose a practical and self-interpretable invasive cancer diagnosis solution. With minimum annotation information, the proposed method mines contrast patterns between normal and malignant images in unsupervised manner and generates a probability map of abnormalities to verify its reasoning. Particularly, a fully convolutional autoencoder is used to learn the dominant structural patterns among normal image patches. Patches that do not share the characteristics of this normal population are detected and analyzed by one-class support vector machine and 1-layer neural network. We apply the proposed method to a public breast cancer image set. Our results, in consultation with a senior pathologist, demonstrate that the proposed method outperforms existing methods. The obtained probability map could benefit the pathology practice by providing visualized verification data and potentially leads to a better understanding of data-driven diagnosis solutions.
Beating SGD Saturation with Tail-Averaging and Minibatching
Authors: Nicole Mücke, Gergely Neu, Lorenzo Rosasco
Date: 2019-02-22 00:00:00
Proceeding: NeurIPS 2019 12

While stochastic gradient descent (SGD) is one of the major workhorses in machine learning, the learning properties of many practically used variants are poorly understood. In this paper, we consider least squares learning in a nonparametric setting and contribute to filling this gap by focusing on the effect and interplay of multiple passes, mini-batching and averaging, and in particular tail averaging. Our results show how these different variants of SGD can be combined to achieve optimal learning errors, hence providing practical insights. In particular, we show for the first time in the literature that tail averaging allows faster convergence rates than uniform averaging in the nonparametric setting. Finally, we show that a combination of tail-averaging and minibatching allows more aggressive step-size choices than using any one of said components.