pwc-archive/papers-with-abstracts huggingface dataset:
RMDL: Random Multimodel Deep Learning for Classification
Authors: Kamran Kowsari, Mojtaba Heidarysafa, Donald E. Brown, Kiana Jafari Meimandi, Laura E. Barnes
Date: 2018-05-03 00:00:00
Tasks: Classification, Deep Learning, Document Classification, Face Recognition, General Classification, Hierarchical Text Classification of Blurbs (GermEval 2019), Image Classification, Multi-Label Text Classification, Unsupervised Pre-training

The continually increasing number of complex datasets each year necessitates
ever improving machine learning methods for robust and accurate categorization
of these data. This paper introduces Random Multimodel Deep Learning (RMDL): a
new ensemble, deep learning approach for classification. Deep learning models
have achieved state-of-the-art results across many domains. RMDL solves the
problem of finding the best deep learning structure and architecture while
simultaneously improving robustness and accuracy through ensembles of deep
learning architectures. RDML can accept as input a variety data to include
text, video, images, and symbolic. This paper describes RMDL and shows test
results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB,
and 20newsgroup. These test results show that RDML produces consistently better
performance than standard methods over a broad range of data types and
classification problems.
Analysis of Fast Structured Dictionary Learning
Authors: Saiprasad Ravishankar, Anna Ma, Deanna Needell
Date: 2018-05-31 00:00:00
Tasks: Dictionary Learning, Operator learning

Sparsity-based models and techniques have been exploited in many signal processing and imaging applications. Data-driven methods based on dictionary and sparsifying transform learning enable learning rich image features from data, and can outperform analytical models. In particular, alternating optimization algorithms have been popular for learning such models. In this work, we focus on alternating minimization for a specific structured unitary sparsifying operator learning problem, and provide a convergence analysis. While the algorithm converges to the critical points of the problem generally, our analysis establishes under mild assumptions, the local linear convergence of the algorithm to the underlying sparsifying model of the data. Analysis and numerical simulations show that our assumptions hold for standard probabilistic data models. In practice, the algorithm is robust to initialization.
Fully Automated Organ Segmentation in Male Pelvic CT Images
Authors: Anjali Balagopal, Samaneh Kazemifar, Dan Nguyen, Mu-Han Lin, Raquibul Hannan, Amir Owrangi, Steve Jiang
Date: 2018-05-31 00:00:00
Tasks: Image Segmentation, Organ Segmentation, Segmentation, Semantic Segmentation

Accurate segmentation of prostate and surrounding organs at risk is important for prostate cancer radiotherapy treatment planning. We present a fully automated workflow for male pelvic CT image segmentation using deep learning. The architecture consists of a 2D localization network followed by a 3D segmentation network for volumetric segmentation of prostate, bladder, rectum, and femoral heads. We used a multi-channel 2D U-Net followed by a 3D U-Net with encoding arm modified with aggregated residual networks, known as ResNeXt. The models were trained and tested on a pelvic CT image dataset comprising 136 patients. Test results show that 3D U-Net based segmentation achieves mean (SD) Dice coefficient values of 90 (2.0)% ,96 (3.0)%, 95 (1.3)%, 95 (1.5)%, and 84 (3.7)% for prostate, left femoral head, right femoral head, bladder, and rectum, respectively, using the proposed fully automated segmentation method.
Whole Brain Susceptibility Mapping Using Harmonic Incompatibility Removal
Authors: Chenglong Bao, Jae Kyu Choi, Bin Dong
Date: 2018-05-31 00:00:00

Quantitative susceptibility mapping (QSM) aims to visualize the three
dimensional susceptibility distribution by solving the field-to-source inverse
problem using the phase data in magnetic resonance signal. However, the inverse
problem is ill-posed since the Fourier transform of integral kernel has zeroes
in the frequency domain. Although numerous regularization based models have
been proposed to overcome this problem, the incompatibility in the field data
has not received enough attention, which leads to deterioration of the
recovery. In this paper, we show that the data acquisition process of QSM
inherently generates a harmonic incompatibility in the measured local field.
Based on such discovery, we propose a novel regularization based susceptibility
reconstruction model with an additional sparsity based regularization term on
the harmonic incompatibility. Numerical experiments show that the proposed
method achieves better performance than the existing approaches.
Scaling provable adversarial defenses
Authors: Eric Wong, Frank R. Schmidt, Jan Hendrik Metzen, J. Zico Kolter
Date: 2018-05-31 00:00:00
Proceeding: NeurIPS 2018 12

Recent work has developed methods for learning deep network classifiers that
are provably robust to norm-bounded adversarial perturbation; however, these
methods are currently only possible for relatively small feedforward networks.
In this paper, in an effort to scale these approaches to substantially larger
models, we extend previous work in three main directions. First, we present a
technique for extending these training procedures to much more general
networks, with skip connections (such as ResNets) and general nonlinearities;
the approach is fully modular, and can be implemented automatically (analogous
to automatic differentiation). Second, in the specific case of $\ell_\infty$
adversarial perturbations and networks with ReLU nonlinearities, we adopt a
nonlinear random projection for training, which scales linearly in the number
of hidden units (previous approaches scaled quadratically). Third, we show how
to further improve robust error through cascade models. On both MNIST and CIFAR
data sets, we train classifiers that improve substantially on the state of the
art in provable robust adversarial error bounds: from 5.8% to 3.1% on MNIST
(with $\ell_\infty$ perturbations of $\epsilon=0.1$), and from 80% to 36.4% on
CIFAR (with $\ell_\infty$ perturbations of $\epsilon=2/255$). Code for all
experiments in the paper is available at
https://github.com/locuslab/convex_adversarial/.
Cyberattack Detection using Deep Generative Models with Variational Inference
Authors: Sarin E. Chandy, Amin Rasekh, Zachary A. Barker, M. Ehsan Shafiee
Date: 2018-05-31 00:00:00
Tasks: Variational Inference

Recent years have witnessed a rise in the frequency and intensity of
cyberattacks targeted at critical infrastructure systems. This study designs a
versatile, data-driven cyberattack detection platform for infrastructure
systems cybersecurity, with a special demonstration in water sector. A deep
generative model with variational inference autonomously learns normal system
behavior and detects attacks as they occur. The model can process the natural
data in its raw form and automatically discover and learn its representations,
hence augmenting system knowledge discovery and reducing the need for laborious
human engineering and domain expertise. The proposed model is applied to a
simulated cyberattack detection problem involving a drinking water distribution
system subject to programmable logic controller hacks, malicious actuator
activation, and deception attacks. The model is only provided with observations
of the system, such as pump pressure and tank water level reads, and is blind
to the internal structures and workings of the water distribution system. The
simulated attacks are manifested in the model's generated reproduction
probability plot, indicating its ability to discern the attacks. There is,
however, need for improvements in reducing false alarms, especially by
optimizing detection thresholds. Altogether, the results indicate ability of
the model in distinguishing attacks and their repercussions from normal system
operation in water distribution systems, and the promise it holds for
cyberattack detection in other domains.
Accurate pedestrian localization in overhead depth images via Height-Augmented HOG
Authors: Werner Kroneman, Alessandro Corbetta, Federico Toschi
Date: 2018-05-31 00:00:00
Tasks: BIG-bench Machine Learning, Data Augmentation

We tackle the challenge of reliably and automatically localizing pedestrians
in real-life conditions through overhead depth imaging at unprecedented
high-density conditions. Leveraging upon a combination of Histogram of Oriented
Gradients-like feature descriptors, neural networks, data augmentation and
custom data annotation strategies, this work contributes a robust and scalable
machine learning-based localization algorithm, which delivers near-human
localization performance in real-time, even with local pedestrian density of
about 3 ped/m2, a case in which most state-of-the art algorithms degrade
significantly in performance.
Efficacy of regularized multi-task learning based on SVM models
Authors: Shaohan Chen, Zhou Fang, Sijie Lu, Chuanhou Gao
Date: 2018-05-31 00:00:00
Tasks: Multi-Task Learning

This paper investigates the efficacy of a regularized multi-task learning (MTL) framework based on SVM (M-SVM) to answer whether MTL always provides reliable results and how MTL outperforms independent learning. We first find that M-SVM is Bayes risk consistent in the limit of large sample size. This implies that despite the task dissimilarities, M-SVM always produces a reliable decision rule for each task in terms of misclassification error when the data size is large enough. Furthermore, we find that the task-interaction vanishes as the data size goes to infinity, and the convergence rates of M-SVM and its single-task counterpart have the same upper bound. The former suggests that M-SVM cannot improve the limit classifier's performance; based on the latter, we conjecture that the optimal convergence rate is not improved when the task number is fixed. As a novel insight of MTL, our theoretical and experimental results achieved an excellent agreement that the benefit of the MTL methods lies in the improvement of the pre-convergence-rate factor (PCR, to be denoted in Section III) rather than the convergence rate. Moreover, this improvement of PCR factors is more significant when the data size is small.
Robust Gyroscope-Aided Camera Self-Calibration
Authors: Santiago Cortés Reina, Arno Solin, Juho Kannala
Date: 2018-05-31 00:00:00
Tasks: Camera Calibration, Video Stabilization

Camera calibration for estimating the intrinsic parameters and lens
distortion is a prerequisite for various monocular vision applications
including feature tracking and video stabilization. This application paper
proposes a model for estimating the parameters on the fly by fusing gyroscope
and camera data, both readily available in modern day smartphones. The model is
based on joint estimation of visual feature positions, camera parameters, and
the camera pose, the movement of which is assumed to follow the movement
predicted by the gyroscope. Our model assumes the camera movement to be free,
but continuous and differentiable, and individual features are assumed to stay
stationary. The estimation is performed online using an extended Kalman filter,
and it is shown to outperform existing methods in robustness and insensitivity
to initialization. We demonstrate the method using simulated data and empirical
data from an iPad.
Multi-Label Transfer Learning for Multi-Relational Semantic Similarity
Authors: Li Zhang, Steven R. Wilson, Rada Mihalcea
Date: 2018-05-31 00:00:00
Proceeding: SEMEVAL 2019 6
Tasks: Multi-Task Learning, regression, Relation, Semantic Similarity, Semantic Textual Similarity, Transfer Learning

Multi-relational semantic similarity datasets define the semantic relations
between two short texts in multiple ways, e.g., similarity, relatedness, and so
on. Yet, all the systems to date designed to capture such relations target one
relation at a time. We propose a multi-label transfer learning approach based
on LSTM to make predictions for several relations simultaneously and aggregate
the losses to update the parameters. This multi-label regression approach
jointly learns the information provided by the multiple relations, rather than
treating them as separate tasks. Not only does this approach outperform the
single-task approach and the traditional multi-task learning approach, but it
also achieves state-of-the-art performance on all but one relation of the Human
Activity Phrase dataset.
Sequential Attacks on Agents for Long-Term Adversarial Goals
Authors: Edgar Tretschk, Seong Joon Oh, Mario Fritz
Date: 2018-05-31 00:00:00
Tasks: Adversarial Attack, Reinforcement Learning, Reinforcement Learning (RL), Self-Driving Cars

Reinforcement learning (RL) has advanced greatly in the past few years with
the employment of effective deep neural networks (DNNs) on the policy networks.
With the great effectiveness came serious vulnerability issues with DNNs that
small adversarial perturbations on the input can change the output of the
network. Several works have pointed out that learned agents with a DNN policy
network can be manipulated against achieving the original task through a
sequence of small perturbations on the input states. In this paper, we
demonstrate furthermore that it is also possible to impose an arbitrary
adversarial reward on the victim policy network through a sequence of attacks.
Our method involves the latest adversarial attack technique, Adversarial
Transformer Network (ATN), that learns to generate the attack and is easy to
integrate into the policy network. As a result of our attack, the victim agent
is misguided to optimise for the adversarial reward over time. Our results
expose serious security threats for RL applications in safety-critical systems
including drones, medical analysis, and self-driving cars.
Multi-Layered Gradient Boosting Decision Trees
Authors: Ji Feng, Yang Yu, Zhi-Hua Zhou
Date: 2018-05-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Representation Learning

Multi-layered representation is believed to be the key ingredient of deep
neural networks especially in cognitive tasks like computer vision. While
non-differentiable models such as gradient boosting decision trees (GBDTs) are
the dominant methods for modeling discrete or tabular data, they are hard to
incorporate with such representation learning ability. In this work, we propose
the multi-layered GBDT forest (mGBDTs), with an explicit emphasis on exploring
the ability to learn hierarchical representations by stacking several layers of
regression GBDTs as its building block. The model can be jointly trained by a
variant of target propagation across layers, without the need to derive
back-propagation nor differentiability. Experiments and visualizations
confirmed the effectiveness of the model in terms of performance and
representation learning ability.
Propagating Confidences through CNNs for Sparse Data Regression
Authors: Abdelrahman Eldesokey, Michael Felsberg, Fahad Shahbaz Khan
Date: 2018-05-30 00:00:00
Tasks: Autonomous Driving, Depth Completion, regression

In most computer vision applications, convolutional neural networks (CNNs)
operate on dense image data generated by ordinary cameras. Designing CNNs for
sparse and irregularly spaced input data is still an open problem with numerous
applications in autonomous driving, robotics, and surveillance. To tackle this
challenging problem, we introduce an algebraically-constrained convolution
layer for CNNs with sparse input and demonstrate its capabilities for the scene
depth completion task. We propose novel strategies for determining the
confidence from the convolution operation and propagating it to consecutive
layers. Furthermore, we propose an objective function that simultaneously
minimizes the data error while maximizing the output confidence. Comprehensive
experiments are performed on the KITTI depth benchmark and the results clearly
demonstrate that the proposed approach achieves superior performance while
requiring three times fewer parameters than the state-of-the-art methods.
Moreover, our approach produces a continuous pixel-wise confidence map enabling
information fusion, state inference, and decision support.
Distributed Estimation of Gaussian Correlations
Authors: Uri Hadar, Ofer Shayevitz
Date: 2018-05-31 00:00:00
Tasks: 2k

We study a distributed estimation problem in which two remotely located
parties, Alice and Bob, observe an unlimited number of i.i.d. samples
corresponding to two different parts of a random vector. Alice can send $k$
bits on average to Bob, who in turn wants to estimate the cross-correlation
matrix between the two parts of the vector. In the case where the parties
observe jointly Gaussian scalar random variables with an unknown correlation
$\rho$, we obtain two constructive and simple unbiased estimators attaining a
variance of $(1-\rho^2)/(2k\ln 2)$, which coincides with a known but
non-constructive random coding result of Zhang and Berger. We extend our
approach to the vector Gaussian case, which has not been treated before, and
construct an estimator that is uniformly better than the scalar estimator
applied separately to each of the correlations. We then show that the Gaussian
performance can essentially be attained even when the distribution is
completely unknown. This in particular implies that in the general problem of
distributed correlation estimation, the variance can decay at least as $O(1/k)$
with the number of transmitted bits. This behavior, however, is not tight: we
give an example of a rich family of distributions for which local samples
reveal essentially nothing about the correlations, and where a slightly
modified estimator attains a variance of $2^{-\Omega(k)}$.
Neural Network Acceptability Judgments
Authors: Alex Warstadt, Amanpreet Singh, Samuel R. Bowman
Date: 2018-05-31 00:00:00
Proceeding: TACL 2019 3
Tasks: CoLA, General Classification, Language Acquisition, Linguistic Acceptability, Sentence

This paper investigates the ability of artificial neural networks to judge the grammatical acceptability of a sentence, with the goal of testing their linguistic competence. We introduce the Corpus of Linguistic Acceptability (CoLA), a set of 10,657 English sentences labeled as grammatical or ungrammatical from published linguistics literature. As baselines, we train several recurrent neural network models on acceptability classification, and find that our models outperform unsupervised models by Lau et al (2016) on CoLA. Error-analysis on specific grammatical phenomena reveals that both Lau et al.'s models and ours learn systematic generalizations like subject-verb-object order. However, all models we test perform far below human level on a wide range of grammatical constructions.
A Method Based on Convex Cone Model for Image-Set Classification with CNN Features
Authors: Naoya Sogi, Taku Nakayama, Kazuhiro Fukui
Date: 2018-05-31 00:00:00
Tasks: Classification, General Classification

In this paper, we propose a method for image-set classification based on
convex cone models, focusing on the effectiveness of convolutional neural
network (CNN) features as inputs. CNN features have non-negative values when
using the rectified linear unit as an activation function. This naturally leads
us to model a set of CNN features by a convex cone and measure the geometric
similarity of convex cones for classification. To establish this framework, we
sequentially define multiple angles between two convex cones by repeating the
alternating least squares method and then define the geometric similarity
between the cones using the obtained angles. Moreover, to enhance our method,
we introduce a discriminant space, maximizing the between-class variance (gaps)
and minimizes the within-class variance of the projected convex cones onto the
discriminant space, similar to a Fisher discriminant analysis. Finally,
classification is based on the similarity between projected convex cones. The
effectiveness of the proposed method was demonstrated experimentally using a
private, multi-view hand shape dataset and two public databases.
On GANs and GMMs
Authors: Eitan Richardson, Yair Weiss
Date: 2018-05-31 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Image Generation

A longstanding problem in machine learning is to find unsupervised methods
that can learn the statistical structure of high dimensional signals. In recent
years, GANs have gained much attention as a possible solution to the problem,
and in particular have shown the ability to generate remarkably realistic high
resolution sampled images. At the same time, many authors have pointed out that
GANs may fail to model the full distribution ("mode collapse") and that using
the learned models for anything other than generating samples may be very
difficult. In this paper, we examine the utility of GANs in learning
statistical models of images by comparing them to perhaps the simplest
statistical model, the Gaussian Mixture Model. First, we present a simple
method to evaluate generative models based on relative proportions of samples
that fall into predetermined bins. Unlike previous automatic methods for
evaluating models, our method does not rely on an additional neural network nor
does it require approximating intractable computations. Second, we compare the
performance of GANs to GMMs trained on the same datasets. While GMMs have
previously been shown to be successful in modeling small patches of images, we
show how to train them on full sized images despite the high dimensionality.
Our results show that GMMs can generate realistic samples (although less sharp
than those of GANs) but also capture the full distribution, which GANs fail to
do. Furthermore, GMMs allow efficient inference and explicit representation of
the underlying statistical structure. Finally, we discuss how GMMs can be used
to generate sharp images.
Policy Search in Continuous Action Domains: an Overview
Authors: Olivier Sigaud, Freek Stulp
Date: 2018-03-13 00:00:00
Tasks: Bayesian Optimization, Deep Reinforcement Learning, Evolutionary Algorithms, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Continuous action policy search is currently the focus of intensive research, driven both by the recent success of deep reinforcement learning algorithms and the emergence of competitors based on evolutionary algorithms. In this paper, we present a broad survey of policy search methods, providing a unified perspective on very different approaches, including also Bayesian Optimization and directed exploration methods. The main message of this overview is in the relationship between the families of methods, but we also outline some factors underlying sample efficiency properties of the various approaches.
Towards a new system for drowsiness detection based on eye blinking and head posture estimation
Authors: M. Ben Dkhil, A. Wali, Adel M. ALIMI
Date: 2018-05-31 00:00:00

Driver drowsiness problem is considered as one of the most important reasons
that increases road accidents number. We propose in this paper a new approach
for realtime driver drowsiness in order to prevent road accidents. The system
uses a smart video camera that takes drivers faces images and supervises the
eye blink (open and close) state and head posture to detect the different
drowsiness states. Face and eye detection are done by Viola and Jones
technique.
Chinese NER Using Lattice LSTM
Authors: Yue Zhang, Jie Yang
Date: 2018-05-05 00:00:00
Proceeding: ACL 2018 7
Tasks: Chinese Named Entity Recognition, NER, Sentence

We investigate a lattice-structured LSTM model for Chinese NER, which encodes
a sequence of input characters as well as all potential words that match a
lexicon. Compared with character-based methods, our model explicitly leverages
word and word sequence information. Compared with word-based methods, lattice
LSTM does not suffer from segmentation errors. Gated recurrent cells allow our
model to choose the most relevant characters and words from a sentence for
better NER results. Experiments on various datasets show that lattice LSTM
outperforms both word-based and character-based LSTM baselines, achieving the
best results.