pwc-archive/papers-with-abstracts huggingface dataset:
Newton Methods for Convolutional Neural Networks
Authors: Chien-Chih Wang, Kent Loong Tan, Chih-Jen Lin
Date: 2018-11-14 00:00:00

Deep learning involves a difficult non-convex optimization problem, which is
often solved by stochastic gradient (SG) methods. While SG is usually
effective, it may not be robust in some situations. Recently, Newton methods
have been investigated as an alternative optimization technique, but nearly all
existing studies consider only fully-connected feedforward neural networks.
They do not investigate other types of networks such as Convolutional Neural
Networks (CNN), which are more commonly used in deep-learning applications. One
reason is that Newton methods for CNN involve complicated operations, and so
far no works have conducted a thorough investigation. In this work, we give
details of all building blocks including function, gradient, and Jacobian
evaluation, and Gauss-Newton matrix-vector products. These basic components are
very important because with them further developments of Newton methods for CNN
become possible. We show that an efficient MATLAB implementation can be done in
just several hundred lines of code and demonstrate that the Newton method gives
competitive test accuracy.
Automatic Grammar Augmentation for Robust Voice Command Recognition
Authors: Yang Yang, Anusha Lalitha, Jinwon Lee, Chris Lott
Date: 2018-11-14 00:00:00

This paper proposes a novel pipeline for automatic grammar augmentation that
provides a significant improvement in the voice command recognition accuracy
for systems with small footprint acoustic model (AM). The improvement is
achieved by augmenting the user-defined voice command set, also called grammar
set, with alternate grammar expressions. For a given grammar set, a set of
potential grammar expressions (candidate set) for augmentation is constructed
from an AM-specific statistical pronunciation dictionary that captures the
consistent patterns and errors in the decoding of AM induced by variations in
pronunciation, pitch, tempo, accent, ambiguous spellings, and noise conditions.
Using this candidate set, greedy optimization based and cross-entropy-method
(CEM) based algorithms are considered to search for an augmented grammar set
with improved recognition accuracy utilizing a command-specific dataset. Our
experiments show that the proposed pipeline along with algorithms considered in
this paper significantly reduce the mis-detection and mis-classification rate
without increasing the false-alarm rate. Experiments also demonstrate the
consistent superior performance of CEM method over greedy-based algorithms.
Unsupervised learning with contrastive latent variable models
Authors: Kristen Severson, Soumya Ghosh, Kenney Ng
Date: 2018-11-14 00:00:00
Tasks: Dimensionality Reduction, feature selection, Subgroup Discovery

In unsupervised learning, dimensionality reduction is an important tool for
data exploration and visualization. Because these aims are typically
open-ended, it can be useful to frame the problem as looking for patterns that
are enriched in one dataset relative to another. These pairs of datasets occur
commonly, for instance a population of interest vs. control or signal vs.
signal free recordings.However, there are few methods that work on sets of data
as opposed to data points or sequences. Here, we present a probabilistic model
for dimensionality reduction to discover signal that is enriched in the target
dataset relative to the background dataset. The data in these sets do not need
to be paired or grouped beyond set membership. By using a probabilistic model
where some structure is shared amongst the two datasets and some is unique to
the target dataset, we are able to recover interesting structure in the latent
space of the target dataset. The method also has the advantages of a
probabilistic model, namely that it allows for the incorporation of prior
information, handles missing data, and can be generalized to different
distributional assumptions. We describe several possible variations of the
model and demonstrate the application of the technique to de-noising, feature
selection, and subgroup discovery settings.
ReSIFT: Reliability-Weighted SIFT-based Image Quality Assessment
Authors: Dogancan Temel, Ghassan AlRegib
Date: 2018-11-14 00:00:00
Tasks: Image Quality Assessment

This paper presents a full-reference image quality estimator based on SIFT
descriptor matching over reliability-weighted feature maps. Reliability
assignment includes a smoothing operation, a transformation to perceptual color
domain, a local normalization stage, and a spectral residual computation with
global normalization. The proposed method ReSIFT is tested on the LIVE and the
LIVE Multiply Distorted databases and compared with 11 state-of-the-art
full-reference quality estimators. In terms of the Pearson and the Spearman
correlation, ReSIFT is the best performing quality estimator in the overall
databases. Moreover, ReSIFT is the best performing quality estimator in at
least one distortion group in compression, noise, and blur category.
Constraint-based Sequential Pattern Mining with Decision Diagrams
Authors: Amin Hosseininasab, Willem-Jan van Hoeve, Andre A. Cire
Date: 2018-11-14 00:00:00
Tasks: Sequential Pattern Mining

Constrained sequential pattern mining aims at identifying frequent patterns
on a sequential database of items while observing constraints defined over the
item attributes. We introduce novel techniques for constraint-based sequential
pattern mining that rely on a multi-valued decision diagram representation of
the database. Specifically, our representation can accommodate multiple item
attributes and various constraint types, including a number of non-monotone
constraints. To evaluate the applicability of our approach, we develop an
MDD-based prefix-projection algorithm and compare its performance against a
typical generate-and-check variant, as well as a state-of-the-art
constraint-based sequential pattern mining algorithm. Results show that our
approach is competitive with or superior to these other methods in terms of
scalability and efficiency.
Learning Optimal Personalized Treatment Rules Using Robust Regression Informed K-NN
Authors: Ruidi Chen, Ioannis Paschalidis
Date: 2018-11-14 00:00:00
Tasks: regression

We develop a prediction-based prescriptive model for learning optimal
personalized treatments for patients based on their Electronic Health Records
(EHRs). Our approach consists of: (i) predicting future outcomes under each
possible therapy using a robustified nonlinear model, and (ii) adopting a
randomized prescriptive policy determined by the predicted outcomes. We show
theoretical results that guarantee the out-of-sample predictive power of the
model, and prove the optimality of the randomized strategy in terms of the
expected true future outcome. We apply the proposed methodology to develop
optimal therapies for patients with type 2 diabetes or hypertension using EHRs
from a major safety-net hospital in New England, and show that our algorithm
leads to a larger reduction of the HbA1c, for diabetics, or systolic blood
pressure, for patients with hypertension, compared to the alternatives. We
demonstrate that our approach outperforms the standard of care under the
robustified nonlinear predictive model.
Interpretable deep learning for guided structure-property explorations in photovoltaics
Authors: Balaji Sesha Sarath Pokuri, Sambuddha Ghosal, Apurva Kokate, Baskar Ganapathysubramanian, Soumik Sarkar
Date: 2018-11-14 00:00:00
Tasks: Deep Learning

The performance of an organic photovoltaic device is intricately connected to
its active layer morphology. This connection between the active layer and
device performance is very expensive to evaluate, either experimentally or
computationally. Hence, designing morphologies to achieve higher performances
is non-trivial and often intractable. To solve this, we first introduce a deep
convolutional neural network (CNN) architecture that can serve as a fast and
robust surrogate for the complex structure-property map. Several tests were
performed to gain trust in this trained model. Then, we utilize this fast
framework to perform robust microstructural design to enhance device
performance.
Spatial Logics and Model Checking for Medical Imaging (Extended Version)
Authors: Fabrizio Banci Buonamici, Gina Belmonte, Vincenzo Ciancia, Diego Latella, Mieke Massink
Date: 2018-11-14 00:00:00
Tasks: Segmentation, Texture Classification

Recent research on spatial and spatio-temporal model checking provides novel
image analysis methodologies, rooted in logical methods for topological spaces.
Medical Imaging (MI) is a field where such methods show potential for
ground-breaking innovation. Our starting point is SLCS, the Spatial Logic for
Closure Spaces -- Closure Spaces being a generalisation of topological spaces,
covering also discrete space structures -- and topochecker, a model-checker for
SLCS (and extensions thereof). We introduce the logical language ImgQL ("Image
Query Language"). ImgQL extends SLCS with logical operators describing distance
and region similarity. The spatio-temporal model checker topochecker is
correspondingly enhanced with state-of-the-art algorithms, borrowed from
computational image processing, for efficient implementation of distancebased
operators, namely distance transforms. Similarity between regions is defined by
means of a statistical similarity operator, based on notions from statistical
texture analysis. We illustrate our approach by means of two examples of
analysis of Magnetic Resonance images: segmentation of glioblastoma and its
oedema, and segmentation of rectal carcinoma.
Minimax Rates in Network Analysis: Graphon Estimation, Community Detection and Hypothesis Testing
Authors: Chao Gao, Zongming Ma
Date: 2018-11-14 00:00:00
Tasks: Community Detection, Graphon Estimation, Two-sample testing

This paper surveys some recent developments in fundamental limits and optimal
algorithms for network analysis. We focus on minimax optimal rates in three
fundamental problems of network analysis: graphon estimation, community
detection, and hypothesis testing. For each problem, we review state-of-the-art
results in the literature followed by general principles behind the optimal
procedures that lead to minimax estimation and testing. This allows us to
connect problems in network analysis to other statistical inference problems
from a general perspective.
Human-like machine learning: limitations and suggestions
Authors: Georgios Mastorakis
Date: 2018-11-14 00:00:00
Tasks: BIG-bench Machine Learning

This paper attempts to address the issues of machine learning in its current
implementation. It is known that machine learning algorithms require a
significant amount of data for training purposes, whereas recent developments
in deep learning have increased this requirement dramatically. The performance
of an algorithm depends on the quality of data and hence, algorithms are as
good as the data they are trained on. Supervised learning is developed based on
human learning processes by analysing named (i.e. annotated) objects, scenes
and actions. Whether training on large quantities of data (i.e. big data) is
the right or the wrong approach, is debatable. The fact is, that training
algorithms the same way we learn ourselves, comes with limitations. This paper
discusses the issues around applying a human-like approach to train algorithms
and the implications of this approach when using limited data. Several current
studies involving non-data-driven algorithms and natural examples are also
discussed and certain alternative approaches are suggested.
A multi-level convolutional LSTM model for the segmentation of left ventricle myocardium in infarcted porcine cine MR images
Authors: Dongqing Zhang, Ilknur Icke, Belma Dogdas, Sarayu Parimal, Smita Sampath, Joseph Forbes, Ansuman Bagchi, Chih-Liang Chin, Antong Chen
Date: 2018-11-14 00:00:00
Tasks: Myocardium Segmentation

Automatic segmentation of left ventricle (LV) myocardium in cardiac
short-axis cine MR images acquired on subjects with myocardial infarction is a
challenging task, mainly because of the various types of image inhomogeneity
caused by the infarctions. Among the approaches proposed to automate the LV
myocardium segmentation task, methods based upon deep convolutional neural
networks (CNN) have demonstrated their exceptional accuracy and robustness in
recent years. However, most of the CNN-based approaches treat the frames in a
cardiac cycle independently, which fails to capture the valuable dynamics of
heart motion. Herein, an approach based on recurrent neural network (RNN),
specifically a multi-level convolutional long short-term memory (ConvLSTM)
model, is proposed to take the motion of the heart into consideration. Based on
a ResNet-56 CNN, LV-related image features in consecutive frames of a cardiac
cycle are extracted at both the low- and high-resolution levels, which are
processed by the corresponding multi-level ConvLSTM models to generate the
myocardium segmentations. A leave-one-out experiment was carried out on a set
of 3,600 cardiac cine MR slices collected in-house for 8 porcine subjects with
surgically induced myocardial infarction. Compared with a solely CNN-based
approach, the proposed approach demonstrated its superior robustness against
image inhomogeneity by incorporating information from adjacent frames. It also
outperformed a one-level ConvLSTM approach thanks to its capabilities to take
advantage of image features at multiple resolution levels.
Looking at the Driver/Rider in Autonomous Vehicles to Predict Take-Over Readiness
Authors: Nachiket Deo, Mohan M. Trivedi
Date: 2018-11-14 00:00:00
Tasks: Autonomous Vehicles

Continuous estimation the driver's take-over readiness is critical for safe
and timely transfer of control during the failure modes of autonomous vehicles.
In this paper, we propose a data-driven approach for estimating the driver's
take-over readiness based purely on observable cues from in-vehicle vision
sensors. We present an extensive naturalistic drive dataset of drivers in a
conditionally autonomous vehicle running on Californian freeways. We collect
subjective ratings for the driver's take-over readiness from multiple human
observers viewing the sensor feed. Analysis of the ratings in terms of
intra-class correlation coefficients (ICCs) shows a high degree of consistency
in the ratings across raters. We define a metric for the driver's take-over
readiness termed the 'Observable Readiness Index (ORI)' based on the ratings.
Finally, we propose an LSTM model for continuous estimation of the driver's ORI
based on a holistic representation of the driver's state, capturing gaze, hand,
pose and foot activity. Our model estimates the ORI with a mean absolute error
of 0.449 on a 5 point scale.
Unsupervised domain adaptation for medical imaging segmentation with self-ensembling
Authors: Christian S. Perone, Pedro Ballester, Rodrigo C. Barros, Julien Cohen-Adad
Date: 2018-11-14 00:00:00
Tasks: Domain Adaptation, Medical Image Segmentation, Semantic Segmentation, Unsupervised Domain Adaptation

Recent advances in deep learning methods have come to define the
state-of-the-art for many medical imaging applications, surpassing even human
judgment in several tasks. Those models, however, when trained to reduce the
empirical risk on a single domain, fail to generalize when applied to other
domains, a very common scenario in medical imaging due to the variability of
images and anatomical structures, even across the same imaging modality. In
this work, we extend the method of unsupervised domain adaptation using
self-ensembling for the semantic segmentation task and explore multiple facets
of the method on a small and realistic publicly-available magnetic resonance
(MRI) dataset. Through an extensive evaluation, we show that self-ensembling
can indeed improve the generalization of the models even when using a small
amount of unlabelled data.
Focus Quality Assessment of High-Throughput Whole Slide Imaging in Digital Pathology
Authors: Mahdi S. Hosseini, Yueyang Zhang, Lyndon Chan, Konstantinos N. Plataniotis, Jasper A. Z. Brawley-Hayes, Savvas Damaskinos
Date: 2018-11-14 00:00:00

One of the challenges facing the adoption of digital pathology workflows for
clinical use is the need for automated quality control. As the scanners
sometimes determine focus inaccurately, the resultant image blur deteriorates
the scanned slide to the point of being unusable. Also, the scanned slide
images tend to be extremely large when scanned at greater or equal 20X image
resolution. Hence, for digital pathology to be clinically useful, it is
necessary to use computational tools to quickly and accurately quantify the
image focus quality and determine whether an image needs to be re-scanned. We
propose a no-reference focus quality assessment metric specifically for digital
pathology images, that operates by using a sum of even-derivative filter bases
to synthesize a human visual system-like kernel, which is modeled as the
inverse of the lens' point spread function. This kernel is then applied to a
digital pathology image to modify high-frequency image information deteriorated
by the scanner's optics and quantify the focus quality at the patch level. We
show in several experiments that our method correlates better with ground-truth
$z$-level data than other methods, and is more computationally efficient. We
also extend our method to generate a local slide-level focus quality heatmap,
which can be used for automated slide quality control, and demonstrate the
utility of our method for clinical scan quality control by comparison with
subjective slide quality scores.
Natural Environment Benchmarks for Reinforcement Learning
Authors: Amy Zhang, Yuxin Wu, Joelle Pineau
Date: 2018-11-14 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

While current benchmark reinforcement learning (RL) tasks have been useful to
drive progress in the field, they are in many ways poor substitutes for
learning with real-world data. By testing increasingly complex RL algorithms on
low-complexity simulation environments, we often end up with brittle RL
policies that generalize poorly beyond the very specific domain. To combat
this, we propose three new families of benchmark RL domains that contain some
of the complexity of the natural world, while still supporting fast and
extensive data acquisition. The proposed domains also permit a characterization
of generalization through fair train/test separation, and easy comparison and
replication of results. Through this work, we challenge the RL research
community to develop more robust algorithms that meet high standards of
evaluation.
A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Tasks
Authors: Victor Sanh, Thomas Wolf, Sebastian Ruder
Date: 2018-11-14 00:00:00
Tasks: Inductive Bias, Multi-Task Learning, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), Relation Extraction

Much effort has been devoted to evaluate whether multi-task learning can be
leveraged to learn rich representations that can be used in various Natural
Language Processing (NLP) down-stream applications. However, there is still a
lack of understanding of the settings in which multi-task learning has a
significant effect. In this work, we introduce a hierarchical model trained in
a multi-task learning setup on a set of carefully selected semantic tasks. The
model is trained in a hierarchical fashion to introduce an inductive bias by
supervising a set of low level tasks at the bottom layers of the model and more
complex tasks at the top layers of the model. This model achieves
state-of-the-art results on a number of tasks, namely Named Entity Recognition,
Entity Mention Detection and Relation Extraction without hand-engineered
features or external NLP tools like syntactic parsers. The hierarchical
training supervision induces a set of shared semantic representations at lower
layers of the model. We show that as we move from the bottom to the top layers
of the model, the hidden states of the layers tend to represent more complex
semantic information.
Verification of Recurrent Neural Networks Through Rule Extraction
Authors: Qinglong Wang, Kaixuan Zhang, Xue Liu, C. Lee Giles
Date: 2018-11-14 00:00:00

The verification problem for neural networks is verifying whether a neural
network will suffer from adversarial samples, or approximating the maximal
allowed scale of adversarial perturbation that can be endured. While most prior
work contributes to verifying feed-forward networks, little has been explored
for verifying recurrent networks. This is due to the existence of a more
rigorous constraint on the perturbation space for sequential data, and the lack
of a proper metric for measuring the perturbation. In this work, we address
these challenges by proposing a metric which measures the distance between
strings, and use deterministic finite automata (DFA) to represent a rigorous
oracle which examines if the generated adversarial samples violate certain
constraints on a perturbation. More specifically, we empirically show that
certain recurrent networks allow relatively stable DFA extraction. As such,
DFAs extracted from these recurrent networks can serve as a surrogate oracle
for when the ground truth DFA is unknown. We apply our verification mechanism
to several widely used recurrent networks on a set of the Tomita grammars. The
results demonstrate that only a few models remain robust against adversarial
samples. In addition, we show that for grammars with different levels of
complexity, there is also a difference in the difficulty of robust learning of
these grammars.
Incentivizing Exploration with Selective Data Disclosure
Authors: Nicole Immorlica, Jieming Mao, Aleksandrs Slivkins, Zhiwei Steven Wu
Date: 2018-11-14 00:00:00
Tasks: Efficient Exploration, Recommendation Systems

We propose and design recommendation systems that incentivize efficient exploration. Agents arrive sequentially, choose actions and receive rewards, drawn from fixed but unknown action-specific distributions. The recommendation system presents each agent with actions and rewards from a subsequence of past agents, chosen ex ante. Thus, the agents engage in sequential social learning, moderated by these subsequences. We asymptotically attain optimal regret rate for exploration, using a flexible frequentist behavioral model and mitigating rationality and commitment assumptions inherent in prior work. We suggest three components of effective recommendation systems: independent focus groups, group aggregators, and interlaced information structures.
Performance Estimation of Synthesis Flows cross Technologies using LSTMs and Transfer Learning
Authors: Cunxi Yu, Wang Zhou
Date: 2018-11-14 00:00:00
Tasks: Transfer Learning

Due to the increasing complexity of Integrated Circuits (ICs) and
System-on-Chip (SoC), developing high-quality synthesis flows within a short
market time becomes more challenging. We propose a general approach that
precisely estimates the Quality-of-Result (QoR), such as delay and area, of
unseen synthesis flows for specific designs. The main idea is training a
Recurrent Neural Network (RNN) regressor, where the flows are inputs and QoRs
are ground truth. The RNN regressor is constructed with Long Short-Term Memory
(LSTM) and fully-connected layers. This approach is demonstrated with 1.2
million data points collected using 14nm, 7nm regular-voltage (RVT), and 7nm
low-voltage (LVT) FinFET technologies with twelve IC designs. The accuracy of
predicting the QoRs (delay and area) within one technology is
$\boldsymbol{\geq}$\textbf{98.0}\% over $\sim$240,000 test points. To enable
accurate predictions cross different technologies and different IC designs, we
propose a transfer-learning approach that utilizes the model pre-trained with
14nm datasets. Our transfer learning approach obtains estimation accuracy
$\geq$96.3\% over $\sim$960,000 test points, using only 100 data points for
training.