pwc-archive/papers-with-abstracts huggingface dataset:
Deep Graph Laplacian Regularization for Robust Denoising of Real Images
Authors: Jin Zeng, Jiahao Pang, Wenxiu Sun, Gene Cheung
Date: 2018-07-31 00:00:00
Tasks: Denoising, Domain Generalization, Image Denoising, Image Restoration

Recent developments in deep learning have revolutionized the paradigm of image restoration. However, its applications on real image denoising are still limited, due to its sensitivity to training data and the complex nature of real image noise. In this work, we combine the robustness merit of model-based approaches and the learning power of data-driven approaches for real image denoising. Specifically, by integrating graph Laplacian regularization as a trainable module into a deep learning framework, we are less susceptible to overfitting than pure CNN-based approaches, achieving higher robustness to small datasets and cross-domain denoising. First, a sparse neighborhood graph is built from the output of a convolutional neural network (CNN). Then the image is restored by solving an unconstrained quadratic programming problem, using a corresponding graph Laplacian regularizer as a prior term. The proposed restoration pipeline is fully differentiable and hence can be end-to-end trained. Experimental results demonstrate that our work is less prone to overfitting given small training data. It is also endowed with strong cross-domain generalization power, outperforming the state-of-the-art approaches by a remarkable margin.
Scaling and bias codes for modeling speaker-adaptive DNN-based speech synthesis systems
Authors: Hieu-Thi Luong, Junichi Yamagishi
Date: 2018-07-31 00:00:00
Tasks: Speech Synthesis

Most neural-network based speaker-adaptive acoustic models for speech
synthesis can be categorized into either layer-based or input-code approaches.
Although both approaches have their own pros and cons, most existing works on
speaker adaptation focus on improving one or the other. In this paper, after we
first systematically overview the common principles of neural-network based
speaker-adaptive models, we show that these approaches can be represented in a
unified framework and can be generalized further. More specifically, we
introduce the use of scaling and bias codes as generalized means for
speaker-adaptive transformation. By utilizing these codes, we can create a more
efficient factorized speaker-adaptive model and capture advantages of both
approaches while reducing their disadvantages. The experiments show that the
proposed method can improve the performance of speaker adaptation compared with
speaker adaptation based on the conventional input code.
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Authors: Mingxing Tan, Bo Chen, Ruoming Pang, Vijay Vasudevan, Mark Sandler, Andrew Howard, Quoc V. Le
Date: 2018-07-31 00:00:00
Proceeding: CVPR 2019 6
Tasks: Image Classification, Neural Architecture Search, object-detection, Object Detection, Real-Time Object Detection

Designing convolutional neural networks (CNN) for mobile devices is challenging because mobile models need to be small and fast, yet still accurate. Although significant efforts have been dedicated to design and improve mobile CNNs on all dimensions, it is very difficult to manually balance these trade-offs when there are so many architectural possibilities to consider. In this paper, we propose an automated mobile neural architecture search (MNAS) approach, which explicitly incorporate model latency into the main objective so that the search can identify a model that achieves a good trade-off between accuracy and latency. Unlike previous work, where latency is considered via another, often inaccurate proxy (e.g., FLOPS), our approach directly measures real-world inference latency by executing the model on mobile phones. To further strike the right balance between flexibility and search space size, we propose a novel factorized hierarchical search space that encourages layer diversity throughout the network. Experimental results show that our approach consistently outperforms state-of-the-art mobile CNN models across multiple vision tasks. On the ImageNet classification task, our MnasNet achieves 75.2% top-1 accuracy with 78ms latency on a Pixel phone, which is 1.8x faster than MobileNetV2 [29] with 0.5% higher accuracy and 2.3x faster than NASNet [36] with 1.2% higher accuracy. Our MnasNet also achieves better mAP quality than MobileNets for COCO object detection. Code is at https://github.com/tensorflow/tpu/tree/master/models/official/mnasnet
Count-Based Exploration with the Successor Representation
Authors: Marlos C. Machado, Marc G. Bellemare, Michael Bowling
Date: 2018-07-31 00:00:00
Proceeding: ICLR 2019 5
Tasks: Atari Games, Efficient Exploration, Reinforcement Learning, Reinforcement Learning (RL)

In this paper we introduce a simple approach for exploration in reinforcement learning (RL) that allows us to develop theoretically justified algorithms in the tabular case but that is also extendable to settings where function approximation is required. Our approach is based on the successor representation (SR), which was originally introduced as a representation defining state generalization by the similarity of successor states. Here we show that the norm of the SR, while it is being learned, can be used as a reward bonus to incentivize exploration. In order to better understand this transient behavior of the norm of the SR we introduce the substochastic successor representation (SSR) and we show that it implicitly counts the number of times each state (or feature) has been observed. We use this result to introduce an algorithm that performs as well as some theoretically sample-efficient approaches. Finally, we extend these ideas to a deep RL algorithm and show that it achieves state-of-the-art performance in Atari 2600 games when in a low sample-complexity regime.
K-medoids Clustering of Data Sequences with Composite Distributions
Authors: Tiexing Wang, Qunwei Li, Donald J. Bucci, Yingbin Liang, Biao Chen, Pramod K. Varshney
Date: 2018-07-31 00:00:00
Tasks: Clustering

This paper studies clustering of data sequences using the k-medoids
algorithm. All the data sequences are assumed to be generated from
\emph{unknown} continuous distributions, which form clusters with each cluster
containing a composite set of closely located distributions (based on a certain
distance metric between distributions). The maximum intra-cluster distance is
assumed to be smaller than the minimum inter-cluster distance, and both values
are assumed to be known. The goal is to group the data sequences together if
their underlying generative distributions (which are unknown) belong to one
cluster. Distribution distance metrics based k-medoids algorithms are proposed
for known and unknown number of distribution clusters. Upper bounds on the
error probability and convergence results in the large sample regime are also
provided. It is shown that the error probability decays exponentially fast as
the number of samples in each data sequence goes to infinity. The error
exponent has a simple form regardless of the distance metric applied when
certain conditions are satisfied. In particular, the error exponent is
characterized when either the Kolmogrov-Smirnov distance or the maximum mean
discrepancy are used as the distance metric. Simulation results are provided to
validate the analysis.
An Enhanced Latent Semantic Analysis Approach for Arabic Document Summarization
Authors: Kamal Al-Sabahi, Zuping Zhang, Jun Long, Khaled Alwesabi
Date: 2018-07-31 00:00:00
Tasks: Document Summarization, Sentence

The fast-growing amount of information on the Internet makes the research in
automatic document summarization very urgent. It is an effective solution for
information overload. Many approaches have been proposed based on different
strategies, such as latent semantic analysis (LSA). However, LSA, when applied
to document summarization, has some limitations which diminish its performance.
In this work, we try to overcome these limitations by applying statistic and
linear algebraic approaches combined with syntactic and semantic processing of
text. First, the part of speech tagger is utilized to reduce the dimension of
LSA. Then, the weight of the term in four adjacent sentences is added to the
weighting schemes while calculating the input matrix to take into account the
word order and the syntactic relations. In addition, a new LSA-based sentence
selection algorithm is proposed, in which the term description is combined with
sentence description for each topic which in turn makes the generated summary
more informative and diverse. To ensure the effectiveness of the proposed
LSA-based sentence selection algorithm, extensive experiment on Arabic and
English are done. Four datasets are used to evaluate the new model, Linguistic
Data Consortium (LDC) Arabic Newswire-a corpus, Essex Arabic Summaries Corpus
(EASC), DUC2002, and Multilingual MSS 2015 dataset. Experimental results on the
four datasets show the effectiveness of the proposed model on Arabic and
English datasets. It performs comprehensively better compared to the
state-of-the-art methods.
Semantic DMN: Formalizing and Reasoning About Decisions in the Presence of Background Knowledge
Authors: Diego Calvanese, Marlon Dumas, Fabrizio Maria Maggi, Marco Montali
Date: 2018-07-31 00:00:00

The Decision Model and Notation (DMN) is a recent OMG standard for the
elicitation and representation of decision models, and for managing their
interconnection with business processes. DMN builds on the notion of decision
tables, and their combination into more complex decision requirements graphs
(DRGs), which bridge between business process models and decision logic models.
DRGs may rely on additional, external business knowledge models, whose
functioning is not part of the standard. In this work, we consider one of the
most important types of business knowledge, namely background knowledge that
conceptually accounts for the structural aspects of the domain of interest, and
propose decision knowledge bases (DKBs), which semantically combine DRGs
modeled in DMN, and domain knowledge captured by means of first-order logic
with datatypes. We provide a logic-based semantics for such an integration, and
formalize different DMN reasoning tasks for DKBs. We then consider background
knowledge formulated as a description logic ontology with datatypes, and show
how the main verification tasks for DMN in this enriched setting can be
formalized as standard DL reasoning services, and actually carried out in
ExpTime. We discuss the effectiveness of our framework on a case study in
maritime security.
Doubly Attentive Transformer Machine Translation
Authors: Hasan Sait Arslan, Mark Fishel, Gholamreza Anbarjafari
Date: 2018-07-30 00:00:00
Tasks: Decoder, Image Captioning, Machine Translation, Multimodal Machine Translation, Translation

In this paper a doubly attentive transformer machine translation model
(DATNMT) is presented in which a doubly-attentive transformer decoder normally
joins spatial visual features obtained via pretrained convolutional neural
networks, conquering any gap between image captioning and translation. In this
framework, the transformer decoder figures out how to take care of
source-language words and parts of an image freely by methods for two separate
attention components in an Enhanced Multi-Head Attention Layer of doubly
attentive transformer, as it generates words in the target language. We find
that the proposed model can effectively exploit not just the scarce multimodal
machine translation data, but also large general-domain text-only machine
translation corpora, or image-text image captioning corpora. The experimental
results show that the proposed doubly-attentive transformer-decoder performs
better than a single-decoder transformer model, and gives the state-of-the-art
results in the English-German multimodal machine translation task.
Fast and Robust Symmetric Image Registration Based on Distances Combining Intensity and Spatial Information
Authors: Johan Öfverstedt, Joakim Lindblad, Nataša Sladoje
Date: 2018-07-30 00:00:00
Tasks: Image Registration, Medical Image Registration

Intensity-based image registration approaches rely on similarity measures to
guide the search for geometric correspondences with high affinity between
images. The properties of the used measure are vital for the robustness and
accuracy of the registration. In this study a symmetric, intensity
interpolation-free, affine registration framework based on a combination of
intensity and spatial information is proposed. The excellent performance of the
framework is demonstrated on a combination of synthetic tests, recovering known
transformations in the presence of noise, and real applications in biomedical
and medical image registration, for both 2D and 3D images. The method exhibits
greater robustness and higher accuracy than similarity measures in common use,
when inserted into a standard gradient-based registration framework available
as part of the open source Insight Segmentation and Registration Toolkit (ITK).
The method is also empirically shown to have a low computational cost, making
it practical for real applications. Source code is available.
Pulse Sequence Resilient Fast Brain Segmentation
Authors: Amod Jog, Bruce Fischl
Date: 2018-07-30 00:00:00
Tasks: Anatomy, Brain Segmentation, Segmentation

Accurate automatic segmentation of brain anatomy from
$T_1$-weighted~($T_1$-w) magnetic resonance images~(MRI) has been a
computationally intensive bottleneck in neuroimaging pipelines, with
state-of-the-art results obtained by unsupervised intensity modeling-based
methods and multi-atlas registration and label fusion. With the advent of
powerful supervised convolutional neural networks~(CNN)-based learning
algorithms, it is now possible to produce a high quality brain segmentation
within seconds. However, the very supervised nature of these methods makes it
difficult to generalize them on data different from what they have been trained
on. Modern neuroimaging studies are necessarily multi-center initiatives with a
wide variety of acquisition protocols. Despite stringent protocol harmonization
practices, it is not possible to standardize the whole gamut of MRI imaging
parameters across scanners, field strengths, receive coils etc., that affect
image contrast. In this paper we propose a CNN-based segmentation algorithm
that, in addition to being highly accurate and fast, is also resilient to
variation in the input $T_1$-w acquisition. Our approach relies on building
approximate forward models of $T_1$-w pulse sequences that produce a typical
test image. We use the forward models to augment the training data with test
data specific training examples. These augmented data can be used to update
and/or build a more robust segmentation model that is more attuned to the test
data imaging properties. Our method generates highly accurate, state-of-the-art
segmentation results~(overall Dice overlap=0.94), within seconds and is
consistent across a wide-range of protocols.
Acquisition of Localization Confidence for Accurate Object Detection
Authors: Borui Jiang, Ruixuan Luo, Jiayuan Mao, Tete Xiao, Yuning Jiang
Date: 2018-07-30 00:00:00
Proceeding: ECCV 2018 9
Tasks: General Classification, Object, object-detection, Object Detection, regression

Modern CNN-based object detectors rely on bounding box regression and
non-maximum suppression to localize objects. While the probabilities for class
labels naturally reflect classification confidence, localization confidence is
absent. This makes properly localized bounding boxes degenerate during
iterative regression or even suppressed during NMS. In the paper we propose
IoU-Net learning to predict the IoU between each detected bounding box and the
matched ground-truth. The network acquires this confidence of localization,
which improves the NMS procedure by preserving accurately localized bounding
boxes. Furthermore, an optimization-based bounding box refinement method is
proposed, where the predicted IoU is formulated as the objective. Extensive
experiments on the MS-COCO dataset show the effectiveness of IoU-Net, as well
as its compatibility with and adaptivity to several state-of-the-art object
detectors.
UH-PRHLT at SemEval-2016 Task 3: Combining Lexical and Semantic-based Features for Community Question Answering
Authors: Marc Franco-Salvador, Sudipta Kar, Thamar Solorio, Paolo Rosso
Date: 2018-07-30 00:00:00
Proceeding: SEMEVAL 2016 6
Tasks: Community Question Answering, Knowledge Graphs, Question Answering

In this work we describe the system built for the three English subtasks of
the SemEval 2016 Task 3 by the Department of Computer Science of the University
of Houston (UH) and the Pattern Recognition and Human Language Technology
(PRHLT) research center - Universitat Polit`ecnica de Val`encia: UH-PRHLT. Our
system represents instances by using both lexical and semantic-based similarity
measures between text pairs. Our semantic features include the use of
distributed representations of words, knowledge graphs generated with the
BabelNet multilingual semantic network, and the FrameNet lexical database.
Experimental results outperform the random and Google search engine baselines
in the three English subtasks. Our approach obtained the highest results of
subtask B compared to the other task participants.
Testing the Efficient Network TRaining (ENTR) Hypothesis: initially reducing training image size makes Convolutional Neural Network training for image recognition tasks more efficient
Authors: Thomas Cherico Wanger, Peter Frohn
Date: 2018-07-30 00:00:00
Tasks: GPU

Convolutional Neural Networks (CNN) for image recognition tasks are seeing
rapid advances in the available architectures and how networks are trained
based on large computational infrastructure and standard datasets with millions
of images. In contrast, performance and time constraints for example, of small
devices and free cloud GPUs necessitate efficient network training (i.e.,
highest accuracy in the shortest inference time possible), often on small
datasets. Here, we hypothesize that initially decreasing image size during
training makes the training process more efficient, because pre-shaping weights
with small images and later utilizing these weights with larger images reduces
initial network parameters and total inference time. We test this Efficient
Network TRaining (ENTR) Hypothesis by training pre-trained Residual Network
(ResNet) models (ResNet18, 34, & 50) on three small datasets (steel
microstructures, bee images, and geographic aerial images) with a free cloud
GPU. Based on three training regimes of i) not, ii) gradually or iii) in one
step increasing image size over the training process, we show that initially
reducing image size increases training efficiency consistently across datasets
and networks. We interpret these results mechanistically in the framework of
regularization theory. Support for the ENTR hypothesis is an important
contribution, because network efficiency improvements for image recognition
tasks are needed for practical applications. In the future, it will be exciting
to see how the ENTR hypothesis holds for large standard datasets like ImageNet
or CIFAR, to better understand the underlying mechanisms, and how these results
compare to other fields such as structural learning.
Deep Recurrent Neural Networks for ECG Signal Denoising
Authors: Karol Antczak
Date: 2018-07-30 00:00:00
Tasks: Denoising, ECG Denoising, Transfer Learning

Electrocardiographic signal is a subject to multiple noises, caused by
various factors. It is therefore a standard practice to denoise such signal
before further analysis. With advances of new branch of machine learning,
called deep learning, new methods are available that promises state-of-the-art
performance for this task. We present a novel approach to denoise
electrocardiographic signals with deep recurrent denoising neural networks. We
utilize a transfer learning technique by pretraining the network using
synthetic data, generated by a dynamic ECG model, and fine-tuning it with a
real data. We also investigate the impact of the synthetic training data on the
network performance on real signals. The proposed method was tested on a real
dataset with varying amount of noise. The results indicate that four-layer deep
recurrent neural network can outperform reference methods for heavily noised
signal. Moreover, networks pretrained with synthetic data seem to have better
results than network trained with real data only. We show that it is possible
to create state-of-the art denoising neural network that, pretrained on
artificial data, can perform exceptionally well on real ECG signals after
proper fine-tuning.
Textual Explanations for Self-Driving Vehicles
Authors: Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, Zeynep Akata
Date: 2018-07-30 00:00:00
Proceeding: ECCV 2018 9

Deep neural perception and control networks have become key components of
self-driving vehicles. User acceptance is likely to benefit from
easy-to-interpret textual explanations which allow end-users to understand what
triggered a particular behavior. Explanations may be triggered by the neural
controller, namely introspective explanations, or informed by the neural
controller's output, namely rationalizations. We propose a new approach to
introspective explanations which consists of two parts. First, we use a visual
(spatial) attention model to train a convolutional network end-to-end from
images to the vehicle control commands, i.e., acceleration and change of
course. The controller's attention identifies image regions that potentially
influence the network's output. Second, we use an attention-based video-to-text
model to produce textual explanations of model actions. The attention maps of
controller and explanation model are aligned so that explanations are grounded
in the parts of the scene that mattered to the controller. We explore two
approaches to attention alignment, strong- and weak-alignment. Finally, we
explore a version of our model that generates rationalizations, and compare
with introspective explanations on the same video segments. We evaluate these
models on a novel driving dataset with ground-truth human explanations, the
Berkeley DeepDrive eXplanation (BDD-X) dataset. Code is available at
https://github.com/JinkyuKimUCB/explainable-deep-driving.
Call Detail Records Driven Anomaly Detection and Traffic Prediction in Mobile Cellular Networks
Authors: Kashif Sultan, Hazrat Ali, Zhongshan Zhang
Date: 2018-07-30 00:00:00
Tasks: Anomaly Detection, BIG-bench Machine Learning, Clustering, Fault Detection, Traffic Prediction

Mobile networks possess information about the users as well as the network.
Such information is useful for making the network end-to-end visible and
intelligent. Big data analytics can efficiently analyze user and network
information, unearth meaningful insights with the help of machine learning
tools. Utilizing big data analytics and machine learning, this work contributes
in three ways. First, we utilize the call detail records (CDR) data to detect
anomalies in the network. For authentication and verification of anomalies, we
use k-means clustering, an unsupervised machine learning algorithm. Through
effective detection of anomalies, we can proceed to suitable design for
resource distribution as well as fault detection and avoidance. Second, we
prepare anomaly-free data by removing anomalous activities and train a neural
network model. By passing anomaly and anomaly-free data through this model, we
observe the effect of anomalous activities in training of the model and also
observe mean square error of anomaly and anomaly free data. Lastly, we use an
autoregressive integrated moving average (ARIMA) model to predict future
traffic for a user. Through simple visualization, we show that anomaly free
data better generalizes the learning models and performs better on prediction
task.