pwc-archive/papers-with-abstracts huggingface dataset:
Combating Adversarial Attacks Using Sparse Representations
Authors: Soorya Gopalakrishnan, Zhinus Marzi, Upamanyu Madhow, Ramtin Pedarsani
Date: 2018-03-11 00:00:00
Tasks: General Classification

It is by now well-known that small adversarial perturbations can induce
classification errors in deep neural networks (DNNs). In this paper, we make
the case that sparse representations of the input data are a crucial tool for
combating such attacks. For linear classifiers, we show that a sparsifying
front end is provably effective against $\ell_{\infty}$-bounded attacks,
reducing output distortion due to the attack by a factor of roughly $K / N$
where $N$ is the data dimension and $K$ is the sparsity level. We then extend
this concept to DNNs, showing that a "locally linear" model can be used to
develop a theoretical foundation for crafting attacks and defenses.
Experimental results for the MNIST dataset show the efficacy of the proposed
sparsifying front end.
Multi Layered-Parallel Graph Convolutional Network (ML-PGCN) for Disease Prediction
Authors: Anees Kazi, Shadi Albarqouni, Karsten Kortuem, Nassir Navab
Date: 2018-04-28 00:00:00
Tasks: Disease Prediction, Prediction

Structural data from Electronic Health Records as complementary information
to imaging data for disease prediction. We incorporate novel weighting layer
into the Graph Convolutional Networks, which weights every element of
structural data by exploring its relation to the underlying disease. We
demonstrate the superiority of our developed technique in terms of
computational speed and obtained encouraging results where our method
outperforms the state-of-the-art methods when applied to two publicly available
datasets ABIDE and Chest X-ray in terms of relative performance for the
accuracy of prediction by 5.31 % and 8.15 % and for the area under the ROC
curve by 4.96 % and 10.36 % respectively. Additionally, the model is
lightweight, fast and easily trainable.
Label-aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition
Authors: Zhenghui Wang, Yanru Qu, Li-Heng Chen, Jian Shen, Wei-Nan Zhang, Shaodian Zhang, Yimei Gao, Gen Gu, Ken Chen, Yong Yu
Date: 2018-04-24 00:00:00
Proceeding: NAACL 2018 6
Tasks: Medical Named Entity Recognition, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Transfer Learning

We study the problem of named entity recognition (NER) from electronic
medical records, which is one of the most fundamental and critical problems for
medical text mining. Medical records which are written by clinicians from
different specialties usually contain quite different terminologies and writing
styles. The difference of specialties and the cost of human annotation makes it
particularly difficult to train a universal medical NER system. In this paper,
we propose a label-aware double transfer learning framework (La-DTL) for
cross-specialty NER, so that a medical NER system designed for one specialty
could be conveniently applied to another one with minimal annotation efforts.
The transferability is guaranteed by two components: (i) we propose label-aware
MMD for feature representation transfer, and (ii) we perform parameter transfer
with a theoretical upper bound which is also label aware. We conduct extensive
experiments on 12 cross-specialty NER tasks. The experimental results
demonstrate that La-DTL provides consistent accuracy improvement over strong
baselines. Besides, the promising experimental results on non-medical NER
scenarios indicate that La-DTL is potential to be seamlessly adapted to a wide
range of NER tasks.
Specifying and Verbalising Answer Set Programs in Controlled Natural Language
Authors: Rolf Schwitter
Date: 2018-04-28 00:00:00
Tasks: Sentence

We show how a bi-directional grammar can be used to specify and verbalise
answer set programs in controlled natural language. We start from a program
specification in controlled natural language and translate this specification
automatically into an executable answer set program. The resulting answer set
program can be modified following certain naming conventions and the revised
version of the program can then be verbalised in the same subset of natural
language that was used as specification language. The bi-directional grammar is
parametrised for processing and generation, deals with referring expressions,
and exploits symmetries in the data structure of the grammar rules whenever
these grammar rules need to be duplicated. We demonstrate that verbalisation
requires sentence planning in order to aggregate similar structures with the
aim to improve the readability of the generated specification. Without
modifications, the generated specification is always semantically equivalent to
the original one; our bi-directional grammar is the first one that allows for
semantic round-tripping in the context of controlled natural language
processing. This paper is under consideration for acceptance in TPLP.
Detect, Quantify, and Incorporate Dataset Bias: A Neuroimaging Analysis on 12,207 Individuals
Authors: Christian Wachinger, Benjamin Gutierrez Becker, Anna Rieckmann
Date: 2018-04-28 00:00:00

Neuroimaging datasets keep growing in size to address increasingly complex
medical questions. However, even the largest datasets today alone are too small
for training complex models or for finding genome wide associations. A solution
is to grow the sample size by merging data across several datasets. However,
bias in datasets complicates this approach and includes additional sources of
variation in the data instead. In this work, we combine 15 large neuroimaging
datasets to study bias. First, we detect bias by demonstrating that scans can
be correctly assigned to a dataset with 73.3% accuracy. Next, we introduce
metrics to quantify the compatibility across datasets and to create embeddings
of neuroimaging sites. Finally, we incorporate the presence of bias for the
selection of a training set for predicting autism. For the quantification of
the dataset bias, we introduce two metrics: the Bhattacharyya distance between
datasets and the age prediction error. The presented embedding of neuroimaging
sites provides an interesting new visualization about the similarity of
different sites. This could be used to guide the merging of data sources, while
limiting the introduction of unwanted variation. Finally, we demonstrate a
clear performance increase when incorporating dataset bias for training set
selection in autism prediction. Overall, we believe that the growing amount of
neuroimaging data necessitates to incorporate data-driven methods for
quantifying dataset bias in future analyses.
Additive Models with Trend Filtering
Authors: Veeranjaneyulu Sadhanala, Ryan J. Tibshirani
Date: 2017-02-16 00:00:00
Tasks: Additive models

We study additive models built with trend filtering, i.e., additive models
whose components are each regularized by the (discrete) total variation of
their $k$th (discrete) derivative, for a chosen integer $k \geq 0$. This
results in $k$th degree piecewise polynomial components, (e.g., $k=0$ gives
piecewise constant components, $k=1$ gives piecewise linear, $k=2$ gives
piecewise quadratic, etc.). Analogous to its advantages in the univariate case,
additive trend filtering has favorable theoretical and computational
properties, thanks in large part to the localized nature of the (discrete)
total variation regularizer that it uses. On the theory side, we derive fast
error rates for additive trend filtering estimates, and show these rates are
minimax optimal when the underlying function is additive and has component
functions whose derivatives are of bounded variation. We also show that these
rates are unattainable by additive smoothing splines (and by additive models
built from linear smoothers, in general). On the computational side, as per the
standard in additive models, backfitting is an appealing method for
optimization, but it is particularly appealing for additive trend filtering
because we can leverage a few highly efficient univariate trend filtering
solvers. Going one step further, we describe a new backfitting algorithm whose
iterations can be run in parallel, which (as far as we know) is the first of
its kind. Lastly, we present experiments to examine the empirical performance
of additive trend filtering.
Approaches for Enriching and Improving Textual Knowledge Bases
Authors: Besnik Fetahu
Date: 2018-04-20 00:00:00
Tasks: Articles, Sentence

Verifiability is one of the core editing principles in Wikipedia, where
editors are encouraged to provide citations for the added statements.
Statements can be any arbitrary piece of text, ranging from a sentence up to a
paragraph. However, in many cases, citations are either outdated, missing, or
link to non-existing references (e.g. dead URL, moved content etc.). In total,
20\% of the cases such citations refer to news articles and represent the
second most cited source. Even in cases where citations are provided, there are
no explicit indicators for the span of a citation for a given piece of text. In
addition to issues related with the verifiability principle, many Wikipedia
entity pages are incomplete, with relevant information that is already
available in online news sources missing. Even for the already existing
citations, there is often a delay between the news publication time and the
reference time.
  In this thesis, we address the aforementioned issues and propose automated
approaches that enforce the verifiability principle in Wikipedia, and suggest
relevant and missing news references for further enriching Wikipedia entity
pages.
Spiking Deep Residual Network
Authors: Yangfan Hu, Huajin Tang, Gang Pan
Date: 2018-04-28 00:00:00

Spiking neural networks (SNNs) have received significant attention for their biological plausibility. SNNs theoretically have at least the same computational power as traditional artificial neural networks (ANNs). They possess potential of achieving energy-efficiency while keeping comparable performance to deep neural networks (DNNs). However, it is still a big challenge to train a very deep SNN. In this paper, we propose an efficient approach to build a spiking version of deep residual network (ResNet). ResNet is considered as a kind of the state-of-the-art convolutional neural networks (CNNs). We employ the idea of converting a trained ResNet to a network of spiking neurons, named Spiking ResNet (S-ResNet). We propose a shortcut conversion model to appropriately scale continuous-valued activations to match firing rates in SNN, and a compensation mechanism to reduce the error caused by discretisation. Experimental results demonstrate that, compared with the state-of-the-art SNN approaches, the proposed Spiking ResNet achieves the best performance on CIFAR-10, CIFAR-100, and ImageNet 2012. Our work is the first time to build a SNN deeper than 40, with comparable performance to ANNs on a large-scale dataset.
Efficient Subpixel Refinement with Symbolic Linear Predictors
Authors: Vincent Lui, Jonathon Geeves, Winston Yii, Tom Drummond
Date: 2018-04-28 00:00:00
Proceeding: CVPR 2018 6

We present an efficient subpixel refinement method usinga learning-based
approach called Linear Predictors. Two key ideas are shown in this paper.
Firstly, we present a novel technique, called Symbolic Linear Predictors, which
makes the learning step efficient for subpixel refinement. This makes our
approach feasible for online applications without compromising accuracy, while
taking advantage of the run-time efficiency of learning based approaches.
Secondly, we show how Linear Predictors can be used to predict the expected
alignment error, allowing us to use only the best keypoints in resource
constrained applications. We show the efficiency and accuracy of our method
through extensive experiments.
Learning Disordered Topological Phases by Statistical Recovery of Symmetry
Authors: Nobuyuki Yoshioka, Yutaka Akagi, Hosho Katsura
Date: 2017-09-18 00:00:00

In this letter, we apply the artificial neural network in a supervised manner
to map out the quantum phase diagram of disordered topological superconductor
in class DIII. Given the disorder that keeps the discrete symmetries of the
ensemble as a whole, translational symmetry which is broken in the
quasiparticle distribution individually is recovered statistically by taking an
ensemble average. By using this, we classify the phases by the artificial
neural network that learned the quasiparticle distribution in the clean limit,
and show that the result is totally consistent with the calculation by the
transfer matrix method or noncommutative geometry approach. If all three
phases, namely the $\mathbb{Z}_2$, trivial, and the thermal metal phases appear
in the clean limit, the machine can classify them with high confidence over the
entire phase diagram. If only the former two phases are present, we find that
the machine remains confused in the certain region, leading us to conclude the
detection of the unknown phase which is eventually identified as the thermal
metal phase. In our method, only the first moment of the quasiparticle
distribution is used for input, but application to a wider variety of systems
is expected by the inclusion of higher moments.
Neural Particle Smoothing for Sampling from Conditional Sequence Models
Authors: Chu-Cheng Lin, Jason Eisner
Date: 2018-04-28 00:00:00
Proceeding: NAACL 2018 6

We introduce neural particle smoothing, a sequential Monte Carlo method for
sampling annotations of an input string from a given probability model. In
contrast to conventional particle filtering algorithms, we train a proposal
distribution that looks ahead to the end of the input string by means of a
right-to-left LSTM. We demonstrate that this innovation can improve the quality
of the sample. To motivate our formal choices, we explain how our neural model
and neural sampler can be viewed as low-dimensional but nonlinear
approximations to working with HMMs over very large state spaces.
Precise Box Score: Extract More Information from Datasets to Improve the Performance of Face Detection
Authors: Ce Qi, Xiaoping Chen, Pingyu Wang, Fei Su
Date: 2018-04-28 00:00:00
Tasks: Face Detection, Model Compression, object-detection, Object Detection

For the training of face detection network based on R-CNN framework, anchors
are assigned to be positive samples if intersection-over-unions (IoUs) with
ground-truth are higher than the first threshold(such as 0.7); and to be
negative samples if their IoUs are lower than the second threshold(such as
0.3). And the face detection model is trained by the above labels. However,
anchors with IoU between first threshold and second threshold are not used. We
propose a novel training strategy, Precise Box Score(PBS), to train object
detection models. The proposed training strategy uses the anchors with IoUs
between the first and second threshold, which can consistently improve the
performance of face detection. Our proposed training strategy extracts more
information from datasets, making better utilization of existing datasets.
What's more, we also introduce a simple but effective model compression
method(SEMCM), which can boost the performance of face detectors further.
Experimental results show that the performance of face detection network can
consistently be improved based on our proposed scheme.
Novel Prediction Techniques Based on Clusterwise Linear Regression
Authors: Igor Gitman, Jieshi Chen, Eric Lei, Artur Dubrawski
Date: 2018-04-28 00:00:00
Tasks: Prediction, regression

In this paper we explore different regression models based on Clusterwise
Linear Regression (CLR). CLR aims to find the partition of the data into $k$
clusters, such that linear regressions fitted to each of the clusters minimize
overall mean squared error on the whole data. The main obstacle preventing to
use found regression models for prediction on the unseen test points is the
absence of a reasonable way to obtain CLR cluster labels when the values of
target variable are unknown. In this paper we propose two novel approaches on
how to solve this problem. The first approach, predictive CLR builds a separate
classification model to predict test CLR labels. The second approach,
constrained CLR utilizes a set of user-specified constraints that enforce
certain points to go to the same clusters. Assuming the constraint values are
known for the test points, they can be directly used to assign CLR labels. We
evaluate these two approaches on three UCI ML datasets as well as on a large
corpus of health insurance claims. We show that both of the proposed algorithms
significantly improve over the known CLR-based regression methods. Moreover,
predictive CLR consistently outperforms linear regression and random forest,
and shows comparable performance to support vector regression on UCI ML
datasets. The constrained CLR approach achieves the best performance on the
health insurance dataset, while enjoying only $\approx 20$ times increased
computational time over linear regression.
Clustrophile 2: Guided Visual Clustering Analysis
Authors: Marco Cavallo, Çağatay Demiralp
Date: 2018-04-09 00:00:00
Tasks: Clustering, Navigate

Data clustering is a common unsupervised learning method frequently used in
exploratory data analysis. However, identifying relevant structures in
unlabeled, high-dimensional data is nontrivial, requiring iterative
experimentation with clustering parameters as well as data features and
instances. The number of possible clusterings for a typical dataset is vast,
and navigating in this vast space is also challenging. The absence of
ground-truth labels makes it impossible to define an optimal solution, thus
requiring user judgment to establish what can be considered a satisfiable
clustering result. Data scientists need adequate interactive tools to
effectively explore and navigate the large clustering space so as to improve
the effectiveness of exploratory clustering analysis. We introduce
\textit{Clustrophile~2}, a new interactive tool for guided clustering analysis.
\textit{Clustrophile~2} guides users in clustering-based exploratory analysis,
adapts user feedback to improve user guidance, facilitates the interpretation
of clusters, and helps quickly reason about differences between clusterings. To
this end, \textit{Clustrophile~2} contributes a novel feature, the Clustering
Tour, to help users choose clustering parameters and assess the quality of
different clustering results in relation to current analysis goals and user
expectations. We evaluate \textit{Clustrophile~2} through a user study with 12
data scientists, who used our tool to explore and interpret sub-cohorts in a
dataset of Parkinson's disease patients. Results suggest that
\textit{Clustrophile~2} improves the speed and effectiveness of exploratory
clustering analysis for both experts and non-experts.
QSAR Classification Modeling for Bioactivity of Molecular Structure via SPL-Logsum
Authors: Liang-Yong Xia, Qing-Yong Wang
Date: 2018-04-23 00:00:00
Tasks: General Classification

Quantitative structure-activity relationship (QSAR) modelling is effective
'bridge' to search the reliable relationship related bioactivity to molecular
structure. A QSAR classification model contains a lager number of redundant,
noisy and irrelevant descriptors. To address this problem, various of methods
have been proposed for descriptor selection. Generally, they can be grouped
into three categories: filters, wrappers, and embedded methods. Regularization
method is an important embedded technology, which can be used for continuous
shrinkage and automatic descriptors selection. In recent years, the interest of
researchers in the application of regularization techniques is increasing in
descriptors selection , such as, logistic regression(LR) with $L_1$ penalty. In
this paper, we proposed a novel descriptor selection method based on self-paced
learning(SPL) with Logsum penalized LR for predicting the bioactivity of
molecular structure. SPL inspired by the learning process of humans and animals
that gradually learns from easy samples(smaller losses) to hard samples(bigger
losses) samples into training and Logsum regularization has capacity to select
few meaningful and significant molecular descriptors, respectively.
Experimental results on simulation and three public QSAR datasets show that our
proposed SPL-Logsum method outperforms other commonly used sparse methods in
terms of classification performance and model interpretation.
Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity
Authors: Xiaohuan Cao, Jianhua Yang, Li Wang, Zhong Xue, Qian Wang, Dinggang Shen
Date: 2018-04-28 00:00:00
Tasks: Image Registration

Non-rigid inter-modality registration can facilitate accurate information
fusion from different modalities, but it is challenging due to the very
different image appearances across modalities. In this paper, we propose to
train a non-rigid inter-modality image registration network, which can directly
predict the transformation field from the input multimodal images, such as CT
and MR images. In particular, the training of our inter-modality registration
network is supervised by intra-modality similarity metric based on the
available paired data, which is derived from a pre-aligned CT and MR dataset.
Specifically, in the training stage, to register the input CT and MR images,
their similarity is evaluated on the warped MR image and the MR image that is
paired with the input CT. So that, the intra-modality similarity metric can be
directly applied to measure whether the input CT and MR images are well
registered. Moreover, we use the idea of dual-modality fashion, in which we
measure the similarity on both CT modality and MR modality. In this way, the
complementary anatomies in both modalities can be jointly considered to more
accurately train the inter-modality registration network. In the testing stage,
the trained inter-modality registration network can be directly applied to
register the new multimodal images without any paired data. Experimental
results have shown that, the proposed method can achieve promising accuracy and
efficiency for the challenging non-rigid inter-modality registration task and
also outperforms the state-of-the-art approaches.
End-to-end Graph-based TAG Parsing with Neural Networks
Authors: Jungo Kasai, Robert Frank, Pauli Xu, William Merrill, Owen Rambow
Date: 2018-04-18 00:00:00
Proceeding: NAACL 2018 6
Tasks: POS, POS Tagging, TAG

We present a graph-based Tree Adjoining Grammar (TAG) parser that uses
BiLSTMs, highway connections, and character-level CNNs. Our best end-to-end
parser, which jointly performs supertagging, POS tagging, and parsing,
outperforms the previously reported best results by more than 2.2 LAS and UAS
points. The graph-based parsing architecture allows for global inference and
rich feature representations for TAG parsing, alleviating the fundamental
trade-off between transition-based and graph-based parsing systems. We also
demonstrate that the proposed parser achieves state-of-the-art performance in
the downstream tasks of Parsing Evaluation using Textual Entailments (PETE) and
Unbounded Dependency Recovery. This provides further support for the claim that
TAG is a viable formalism for problems that require rich structural analysis of
sentences.
Segmentation-Free Approaches for Handwritten Numeral String Recognition
Authors: Andre G. Hochuli, Luiz E. S. Oliveira, Alceu S. Britto Jr, Robert Sabourin
Date: 2018-04-24 00:00:00
Tasks: Segmentation

This paper presents segmentation-free strategies for the recognition of
handwritten numeral strings of unknown length. A synthetic dataset of touching
numeral strings of sizes 2-, 3- and 4-digits was created to train end-to-end
solutions based on Convolutional Neural Networks. A robust experimental
protocol is used to show that the proposed segmentation-free methods may reach
the state-of-the-art performance without suffering the heavy burden of
over-segmentation based methods. In addition, they confirmed the importance of
introducing contextual information in the design of end-to-end solutions, such
as the proposed length classifier when recognizing numeral strings.