pwc-archive/papers-with-abstracts huggingface dataset:
Provable Robustness of ReLU networks via Maximization of Linear Regions
Authors: Francesco Croce, Maksym Andriushchenko, Matthias Hein
Date: 2018-10-17 00:00:00

It has been shown that neural network classifiers are not robust. This raises
concerns about their usage in safety-critical systems. We propose in this paper
a regularization scheme for ReLU networks which provably improves the
robustness of the classifier by maximizing the linear regions of the classifier
as well as the distance to the decision boundary. Our techniques allow even to
find the minimal adversarial perturbation for a fraction of test points for
large networks. In the experiments we show that our approach improves upon
adversarial training both in terms of lower and upper bounds on the robustness
and is comparable or better than the state-of-the-art in terms of test error
and robustness.
Hierarchical Methods of Moments
Authors: Matteo Ruffini, Guillaume Rabusseau, Borja Balle
Date: 2018-10-17 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Tensor Decomposition

Spectral methods of moments provide a powerful tool for learning the
parameters of latent variable models. Despite their theoretical appeal, the
applicability of these methods to real data is still limited due to a lack of
robustness to model misspecification. In this paper we present a hierarchical
approach to methods of moments to circumvent such limitations. Our method is
based on replacing the tensor decomposition step used in previous algorithms
with approximate joint diagonalization. Experiments on topic modeling show that
our method outperforms previous tensor decomposition methods in terms of speed
and model quality.
What might matter in autonomous cars adoption: first person versus third person scenarios
Authors: Eva Zackova, Jan Romportl
Date: 2018-10-17 00:00:00

The discussion between the automotive industry, governments, ethicists,
policy makers and general public about autonomous cars' moral agency is
widening, and therefore we see the need to bring more insight into what
meta-factors might actually influence the outcomes of such discussions, surveys
and plebiscites. In our study, we focus on the psychological (personality
traits), practical (active driving experience), gender and rhetoric/framing
factors that might impact and even determine respondents' a priori preferences
of autonomous cars' operation. We conducted an online survey (N=430) to collect
data that show that the third person scenario is less biased than the first
person scenario when presenting ethical dilemma related to autonomous cars.
According to our analysis, gender bias should be explored in more extensive
future studies as well. We recommend any participatory technology assessment
discourse to use the third person scenario and to direct attention to the way
any autonomous car related debate is introduced, especially in terms of
linguistic and communication aspects and gender.
Word Embeddings from Large-Scale Greek Web Content
Authors: Stamatis Outsios, Konstantinos Skianis, Polykarpos Meladianos, Christos Xypolopoulos, Michalis Vazirgiannis
Date: 2018-10-08 00:00:00
Tasks: Word Embeddings

Word embeddings are undoubtedly very useful components in many NLP tasks. In
this paper, we present word embeddings and other linguistic resources trained
on the largest to date digital Greek language corpus. We also present a live
web tool for testing the Greek word embeddings, by offering "analogy",
"similarity score" and "most similar words" functions. Through our explorer,
one could interact with the Greek word vectors.
Reconstructing networks with unknown and heterogeneous errors
Authors: Tiago P. Peixoto
Date: 2018-06-09 00:00:00
Tasks: Bayesian Inference, Stochastic Block Model

The vast majority of network datasets contains errors and omissions, although
this is rarely incorporated in traditional network analysis. Recently, an
increasing effort has been made to fill this methodological gap by developing
network reconstruction approaches based on Bayesian inference. These
approaches, however, rely on assumptions of uniform error rates and on direct
estimations of the existence of each edge via repeated measurements, something
that is currently unavailable for the majority of network data. Here we develop
a Bayesian reconstruction approach that lifts these limitations by not only
allowing for heterogeneous errors, but also for single edge measurements
without direct error estimates. Our approach works by coupling the inference
approach with structured generative network models, which enable the
correlations between edges to be used as reliable uncertainty estimates.
Although our approach is general, we focus on the stochastic block model as the
basic generative process, from which efficient nonparametric inference can be
performed, and yields a principled method to infer hierarchical community
structure from noisy data. We demonstrate the efficacy of our approach with a
variety of empirical and artificial networks.
Exploring Textual and Speech information in Dialogue Act Classification with Speaker Domain Adaptation
Authors: Xuanli He, Quan Hung Tran, William Havard, Laurent Besacier, Ingrid Zukerman, Gholamreza Haffari
Date: 2018-10-17 00:00:00
Proceeding: ALTA 2018 12
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Classification, Dialogue Act Classification, Domain Adaptation, General Classification, speech-recognition, Speech Recognition

In spite of the recent success of Dialogue Act (DA) classification, the
majority of prior works focus on text-based classification with oracle
transcriptions, i.e. human transcriptions, instead of Automatic Speech
Recognition (ASR)'s transcriptions. In spoken dialog systems, however, the
agent would only have access to noisy ASR transcriptions, which may further
suffer performance degradation due to domain shift. In this paper, we explore
the effectiveness of using both acoustic and textual signals, either oracle or
ASR transcriptions, and investigate speaker domain adaptation for DA
classification. Our multimodal model proves to be superior to the unimodal
models, particularly when the oracle transcriptions are not available. We also
propose an effective method for speaker domain adaptation, which achieves
competitive results.
Generalized Earthquake Frequency-Magnitude Distribution Described by Asymmetric Laplace Mixture Modelling
Authors: Arnaud Mignan
Date: 2018-10-17 00:00:00

The complete part of the earthquake frequency-magnitude distribution (FMD),
above completeness magnitude mc, is well described by the Gutenberg-Richter
law. The parameter mc however varies in space due to the seismic network
configuration, yielding a convoluted FMD shape below max(mc). This paper
investigates the shape of the generalized FMD (GFMD), which may be described as
a mixture of elemental FMDs (eFMDs) defined as asymmetric Laplace distributions
of mode mc [Mignan, 2012, https://doi.org/10.1029/2012JB009347]. An asymmetric
Laplace mixture model (GFMD- ALMM) is thus proposed with its parameters
(detection parameter kappa, Gutenberg-Richter beta-value, mc distribution, as
well as number K and weight w of eFMD components) estimated using a
semi-supervised hard expectation maximization approach including BIC penalties
for model complexity. The performance of the proposed method is analysed, with
encouraging results obtained: kappa, beta, and the mc distribution range are
retrieved for different GFMD shapes in simulations, as well as in regional
catalogues (southern and northern California, Nevada, Taiwan, France), in a
global catalogue, and in an aftershock sequence (Christchurch, New Zealand). We
find max(mc) to be conservative compared to other methods, kappa = k/log(10) =
3 in most catalogues (compared to beta = b/log(10) = 1), but also that biases
in kappa and beta may occur when rounding errors are present below
completeness. The GFMD-ALMM, by modelling different FMD shapes in an autonomous
manner, opens the door to new statistical analyses in the realm of incomplete
seismicity data, which could in theory improve earthquake forecasting by
considering c. ten times more events.
EMHMM Simulation Study
Authors: Antoni B. Chan, Janet H. Hsiao
Date: 2018-10-17 00:00:00
Tasks: Bayesian Inference

Eye Movement analysis with Hidden Markov Models (EMHMM) is a method for modeling eye fixation sequences using hidden Markov models (HMMs). In this report, we run a simulation study to investigate the estimation error for learning HMMs with variational Bayesian inference, with respect to the number of sequences and the sequence lengths. We also relate the estimation error measured by KL divergence and L1-norm to a corresponding distortion in the ground-truth HMM parameters.
Learning to quantify emphysema extent: What labels do we need?
Authors: Silas Nyboe Ørting, Jens Petersen, Laura H. Thomsen, Mathilde M. W. Wille, Marleen de Bruijne
Date: 2018-10-17 00:00:00
Tasks: BIG-bench Machine Learning, Multiple Instance Learning

Accurate assessment of pulmonary emphysema is crucial to assess disease
severity and subtype, to monitor disease progression and to predict lung cancer
risk. However, visual assessment is time-consuming and subject to substantial
inter-rater variability and standard densitometry approaches to quantify
emphysema remain inferior to visual scoring. We explore if machine learning
methods that learn from a large dataset of visually assessed CT scans can
provide accurate estimates of emphysema extent. We further investigate if
machine learning algorithms that learn from a scoring of emphysema extent can
outperform algorithms that learn only from a scoring of emphysema presence. We
compare four Multiple Instance Learning classifiers that are trained on
emphysema presence labels, and five Learning with Label Proportions classifiers
that are trained on emphysema extent labels. We evaluate performance on 600
low-dose CT scans from the Danish Lung Cancer Screening Trial and find that
learning from emphysema presence labels, which are much easier to obtain, gives
equally good performance to learning from emphysema extent labels. The best
classifiers achieve intra-class correlation coefficients around 0.90 and
average overall agreement with raters of 78% and 79% on six emphysema extent
classes versus inter-rater agreement of 83%.
Learning an MR acquisition-invariant representation using Siamese neural networks
Authors: Wouter M. Kouw, Marco Loog, Wilbert Bartels, Adriënne M. Mendrik
Date: 2018-10-17 00:00:00

Generalization of voxelwise classifiers is hampered by differences between
MRI-scanners, e.g. different acquisition protocols and field strengths. To
address this limitation, we propose a Siamese neural network (MRAI-NET) that
extracts acquisition-invariant feature vectors. These can consequently be used
by task-specific methods, such as voxelwise classifiers for tissue
segmentation. MRAI-NET is tested on both simulated and real patient data.
Experiments show that MRAI-NET outperforms voxelwise classifiers trained on the
source or target scanner data when a small number of labeled samples is
available.
Embarrassingly Simple Model for Early Action Proposal
Authors: Marcos Baptista-Ríos, Roberto J. López-Sastre, Franciso Javier Acevedo-Rodríguez, Saturnino Maldonado-Bascón
Date: 2018-10-17 00:00:00
Tasks: model

Early action proposal consists in generating high quality candidate temporal
segments that are likely to contain an action in a video stream, as soon as
they happen. Many sophisticated approaches have been proposed for the action
proposal problem but from the off-line perspective. On the contrary, we focus
on the on-line version of the problem, proposing a simple classifier-based
model, using standard 3D CNNs, that performs significantly better than the
state of the art.
Continual Learning of Recurrent Neural Networks by Locally Aligning Distributed Representations
Authors: Alexander Ororbia, Ankur Mali, C. Lee Giles, Daniel Kifer
Date: 2018-10-17 00:00:00
Tasks: Continual Learning, Language Modeling, Language Modelling, Rolling Shutter Correction

Temporal models based on recurrent neural networks have proven to be quite powerful in a wide variety of applications. However, training these models often relies on back-propagation through time, which entails unfolding the network over many time steps, making the process of conducting credit assignment considerably more challenging. Furthermore, the nature of back-propagation itself does not permit the use of non-differentiable activation functions and is inherently sequential, making parallelization of the underlying training process difficult. Here, we propose the Parallel Temporal Neural Coding Network (P-TNCN), a biologically inspired model trained by the learning algorithm we call Local Representation Alignment. It aims to resolve the difficulties and problems that plague recurrent networks trained by back-propagation through time. The architecture requires neither unrolling in time nor the derivatives of its internal activation functions. We compare our model and learning procedure to other back-propagation through time alternatives (which also tend to be computationally expensive), including real-time recurrent learning, echo state networks, and unbiased online recurrent optimization. We show that it outperforms these on sequence modeling benchmarks such as Bouncing MNIST, a new benchmark we denote as Bouncing NotMNIST, and Penn Treebank. Notably, our approach can in some instances outperform full back-propagation through time as well as variants such as sparse attentive back-tracking. Significantly, the hidden unit correction phase of P-TNCN allows it to adapt to new datasets even if its synaptic weights are held fixed (zero-shot adaptation) and facilitates retention of prior generative knowledge when faced with a task sequence. We present results that show the P-TNCN's ability to conduct zero-shot adaptation and online continual sequence modeling.
Adversarial Balancing for Causal Inference
Authors: Michal Ozery-Flato, Pierre Thodoroff, Matan Ninio, Michal Rosen-Zvi, Tal El-Hay
Date: 2018-10-17 00:00:00
Tasks: Causal Inference

Biases in observational data of treatments pose a major challenge to estimating expected treatment outcomes in different populations. An important technique that accounts for these biases is reweighting samples to minimize the discrepancy between treatment groups. We present a novel reweighting approach that uses bi-level optimization to alternately train a discriminator to minimize classification error, and a balancing weights generator that uses exponentiated gradient descent to maximize this error. This approach borrows principles from generative adversarial networks (GANs) to exploit the power of classifiers for measuring two-sample divergence. We provide theoretical results for conditions in which the estimation error is bounded by two factors: (i) the discrepancy measure induced by the discriminator; and (ii) the weights variability. Experimental results on several benchmarks comparing to previous state-of-the-art reweighting methods demonstrate the effectiveness of this approach in estimating causal effects.
Recognizing Partial Biometric Patterns
Authors: Lingxiao He, Zhenan Sun, Yuhao Zhu, Yunbo Wang
Date: 2018-10-17 00:00:00
Tasks: Dictionary Learning, Face Recognition, Person Re-Identification, Triplet

Biometric recognition on partial captured targets is challenging, where only
several partial observations of objects are available for matching. In this
area, deep learning based methods are widely applied to match these partial
captured objects caused by occlusions, variations of postures or just partial
out of view in person re-identification and partial face recognition. However,
most current methods are not able to identify an individual in case that some
parts of the object are not obtainable, while the rest are specialized to
certain constrained scenarios. To this end, we propose a robust general
framework for arbitrary biometric matching scenarios without the limitations of
alignment as well as the size of inputs. We introduce a feature post-processing
step to handle the feature maps from FCN and a dictionary learning based
Spatial Feature Reconstruction (SFR) to match different sized feature maps in
this work. Moreover, the batch hard triplet loss function is applied to
optimize the model. The applicability and effectiveness of the proposed method
are demonstrated by the results from experiments on three person
re-identification datasets (Market1501, CUHK03, DukeMTMC-reID), two partial
person datasets (Partial REID and Partial iLIDS) and two partial face datasets
(CASIA-NIR-Distance and Partial LFW), on which state-of-the-art performance is
ensured in comparison with several state-of-the-art approaches. The code is
released online and can be found on the website:
https://github.com/lingxiao-he/Partial-Person-ReID.
Sequence to Sequence Mixture Model for Diverse Machine Translation
Authors: Xuanli He, Gholamreza Haffari, Mohammad Norouzi
Date: 2018-10-17 00:00:00
Proceeding: CONLL 2018 10
Tasks: Clustering, Diversity, Machine Translation, Translation

Sequence to sequence (SEQ2SEQ) models often lack diversity in their generated
translations. This can be attributed to the limitation of SEQ2SEQ models in
capturing lexical and syntactic variations in a parallel corpus resulting from
different styles, genres, topics, or ambiguity of the translation process. In
this paper, we develop a novel sequence to sequence mixture (S2SMIX) model that
improves both translation diversity and quality by adopting a committee of
specialized translation models rather than a single translation model. Each
mixture component selects its own training dataset via optimization of the
marginal loglikelihood, which leads to a soft clustering of the parallel
corpus. Experiments on four language pairs demonstrate the superiority of our
mixture model compared to a SEQ2SEQ baseline with standard or diversity-boosted
beam search. Our mixture model uses negligible additional parameters and incurs
no extra computation cost during decoding.
Analysis of Railway Accidents' Narratives Using Deep Learning
Authors: Mojtaba Heidarysafa, Kamran Kowsari, Laura E. Barnes, Donald E. Brown
Date: 2018-10-17 00:00:00
Tasks: Deep Learning, Word Embeddings

Automatic understanding of domain specific texts in order to extract useful relationships for later use is a non-trivial task. One such relationship would be between railroad accidents' causes and their correspondent descriptions in reports. From 2001 to 2016 rail accidents in the U.S. cost more than $4.6B. Railroads involved in accidents are required to submit an accident report to the Federal Railroad Administration (FRA). These reports contain a variety of fixed field entries including primary cause of the accidents (a coded variable with 389 values) as well as a narrative field which is a short text description of the accident. Although these narratives provide more information than a fixed field entry, the terminologies used in these reports are not easy to understand by a non-expert reader. Therefore, providing an assisting method to fill in the primary cause from such domain specific texts(narratives) would help to label the accidents with more accuracy. Another important question for transportation safety is whether the reported accident cause is consistent with narrative description. To address these questions, we applied deep learning methods together with powerful word embeddings such as Word2Vec and GloVe to classify accident cause values for the primary cause field using the text in the narratives. The results show that such approaches can both accurately classify accident causes based on report narratives and find important inconsistencies in accident reporting.
Progressive Weight Pruning of Deep Neural Networks using ADMM
Authors: Shaokai Ye, Tianyun Zhang, Kaiqi Zhang, Jiayu Li, Kaidi Xu, Yunfei Yang, Fuxun Yu, Jian Tang, Makan Fardad, Sijia Liu, Xiang Chen, Xue Lin, Yanzhi Wang
Date: 2018-10-17 00:00:00
Proceeding: ICLR 2019 5
Tasks: Edge-computing, Model Compression

Deep neural networks (DNNs) although achieving human-level performance in
many domains, have very large model size that hinders their broader
applications on edge computing devices. Extensive research work have been
conducted on DNN model compression or pruning. However, most of the previous
work took heuristic approaches. This work proposes a progressive weight pruning
approach based on ADMM (Alternating Direction Method of Multipliers), a
powerful technique to deal with non-convex optimization problems with
potentially combinatorial constraints. Motivated by dynamic programming, the
proposed method reaches extremely high pruning rate by using partial prunings
with moderate pruning rates. Therefore, it resolves the accuracy degradation
and long convergence time problems when pursuing extremely high pruning ratios.
It achieves up to 34 times pruning rate for ImageNet dataset and 167 times
pruning rate for MNIST dataset, significantly higher than those reached by the
literature work. Under the same number of epochs, the proposed method also
achieves faster convergence and higher compression rates. The codes and pruned
DNN models are released in the link bit.ly/2zxdlss
XJTLUIndoorLoc: A New Fingerprinting Database for Indoor Localization and Trajectory Estimation Based on Wi-Fi RSS and Geomagnetic Field
Authors: Zhenghang Zhong, Zhe Tang, Xiangxing Li, Tiancheng Yuan, Yang Yang, Meng Wei, Yuanyuan Zhang, Renzhi Sheng, Naomi Grant, Chongfeng Ling, Xintao Huan, Kyeong Soo Kim, Sanghyuk Lee
Date: 2018-10-17 00:00:00
Tasks: Indoor Localization

In this paper, we present a new location fingerprinting database comprised of
Wi-Fi received signal strength (RSS) and geomagnetic field intensity measured
with multiple devices at a multi-floor building in Xi'an Jiatong-Liverpool
University, Suzhou, China. We also provide preliminary results of localization
and trajectory estimation based on convolutional neural network (CNN) and long
short-term memory (LSTM) network with this database. For localization, we map
RSS data for a reference point to an image-like, two-dimensional array and then
apply CNN which is popular in image and video analysis and recognition. For
trajectory estimation, we use a modified random way point model to efficiently
generate continuous step traces imitating human walking and train a stacked
two-layer LSTM network with the generated data to remember the changing pattern
of geomagnetic field intensity against (x,y) coordinates. Experimental results
demonstrate the usefulness of our new database and the feasibility of the CNN
and LSTM-based localization and trajectory estimation with the database.
Simple Regret Minimization for Contextual Bandits
Authors: Aniket Anand Deshmukh, Srinagesh Sharma, James W. Cutler, Mark Moldwin, Clayton Scott
Date: 2018-10-17 00:00:00
Tasks: Multi-Armed Bandits

There are two variants of the classical multi-armed bandit (MAB) problem that have received considerable attention from machine learning researchers in recent years: contextual bandits and simple regret minimization. Contextual bandits are a sub-class of MABs where, at every time step, the learner has access to side information that is predictive of the best arm. Simple regret minimization assumes that the learner only incurs regret after a pure exploration phase. In this work, we study simple regret minimization for contextual bandits. Motivated by applications where the learner has separate training and autonomous modes, we assume that the learner experiences a pure exploration phase, where feedback is received after every action but no regret is incurred, followed by a pure exploitation phase in which regret is incurred but there is no feedback. We present the Contextual-Gap algorithm and establish performance guarantees on the simple regret, i.e., the regret during the pure exploitation phase. Our experiments examine a novel application to adaptive sensor selection for magnetic field estimation in interplanetary spacecraft, and demonstrate considerable improvement over algorithms designed to minimize the cumulative regret.