pwc-archive/papers-with-abstracts huggingface dataset:
Modeling the Biological Pathology Continuum with HSIC-regularized Wasserstein Auto-encoders
Authors: Denny Wu, Hirofumi Kobayashi, Charles Ding, Lei Cheng, Keisuke Goda Marzyeh Ghassemi
Date: 2019-01-20 00:00:00

A crucial challenge in image-based modeling of biomedical data is to identify
trends and features that separate normality and pathology. In many cases, the
morphology of the imaged object exhibits continuous change as it deviates from
normality, and thus a generative model can be trained to model this
morphological continuum. Moreover, given side information that correlates to
certain trend in morphological change, a latent variable model can be
regularized such that its latent representation reflects this side information.
In this work, we use the Wasserstein Auto-encoder to model this pathology
continuum, and apply the Hilbert-Schmitt Independence Criterion (HSIC) to
enforce dependency between certain latent features and the provided side
information. We experimentally show that the model can provide disentangled and
interpretable latent representations and also generate a continuum of
morphological changes that corresponds to change in the side information.
Beyond Turing: Intelligent Agents Centered on the User
Authors: Maxine Eskenazi, Shikib Mehri, Evgeniia Razumovskaia, Tiancheng Zhao
Date: 2019-01-20 00:00:00
Tasks: Chatbot, slot-filling, Slot Filling

Most research on intelligent agents centers on the agent and not on the user.
We look at the origins of agent-centric research for slot-filling, gaming and
chatbot agents. We then argue that it is important to concentrate more on the
user. After reviewing relevant literature, some approaches for creating and
assessing user-centric systems are proposed.
Hierarchical Attentional Hybrid Neural Networks for Document Classification
Authors: Jader Abreu, Luis Fred, David Macêdo, Cleber Zanchettin
Date: 2019-01-20 00:00:00
Tasks: Classification, Document Classification, General Classification, Text Classification

Document classification is a challenging task with important applications. The deep learning approaches to the problem have gained much attention recently. Despite the progress, the proposed models do not incorporate the knowledge of the document structure in the architecture efficiently and not take into account the contexting importance of words and sentences. In this paper, we propose a new approach based on a combination of convolutional neural networks, gated recurrent units, and attention mechanisms for document classification tasks. The main contribution of this work is the use of convolution layers to extract more meaningful, generalizable and abstract features by the hierarchical representation. The proposed method in this paper improves the results of the current attention-based approaches for document classification.
Evaluating Text-to-Image Matching using Binary Image Selection (BISON)
Authors: Hexiang Hu, Ishan Misra, Laurens van der Maaten
Date: 2019-01-19 00:00:00
Tasks: Image Captioning, Image Retrieval, Retrieval

Providing systems the ability to relate linguistic and visual content is one
of the hallmarks of computer vision. Tasks such as text-based image retrieval
and image captioning were designed to test this ability but come with
evaluation measures that have a high variance or are difficult to interpret. We
study an alternative task for systems that match text and images: given a text
query, the system is asked to select the image that best matches the query from
a pair of semantically similar images. The system's accuracy on this Binary
Image SelectiON (BISON) task is interpretable, eliminates the reliability
problems of retrieval evaluations, and focuses on the system's ability to
understand fine-grained visual structure. We gather a BISON dataset that
complements the COCO dataset and use it to evaluate modern text-based image
retrieval and image captioning systems. Our results provide novel insights into
the performance of these systems. The COCO-BISON dataset and corresponding
evaluation code are publicly available from \url{http://hexianghu.com/bison/}.
Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
Authors: Charbel Sakr, Naigang Wang, Chia-Yu Chen, Jungwook Choi, Ankur Agrawal, Naresh Shanbhag, Kailash Gopalakrishnan
Date: 2019-01-19 00:00:00
Proceeding: ICLR 2019 5

Efforts to reduce the numerical precision of computations in deep learning
training have yielded systems that aggressively quantize weights and
activations, yet employ wide high-precision accumulators for partial sums in
inner-product operations to preserve the quality of convergence. The absence of
any framework to analyze the precision requirements of partial sum
accumulations results in conservative design choices. This imposes an
upper-bound on the reduction of complexity of multiply-accumulate units. We
present a statistical approach to analyze the impact of reduced accumulation
precision on deep learning training. Observing that a bad choice for
accumulation precision results in loss of information that manifests itself as
a reduction in variance in an ensemble of partial sums, we derive a set of
equations that relate this variance to the length of accumulation and the
minimum number of bits needed for accumulation. We apply our analysis to three
benchmark networks: CIFAR-10 ResNet 32, ImageNet ResNet 18 and ImageNet
AlexNet. In each case, with accumulation precision set in accordance with our
proposed equations, the networks successfully converge to the single precision
floating-point baseline. We also show that reducing accumulation precision
further degrades the quality of the trained network, proving that our equations
produce tight bounds. Overall this analysis enables precise tailoring of
computation hardware to the application, yielding area- and power-optimal
systems.
Fitting ReLUs via SGD and Quantized SGD
Authors: Seyed Mohammadreza Mousavi Kalan, Mahdi Soltanolkotabi, A. Salman Avestimehr
Date: 2019-01-19 00:00:00

In this paper we focus on the problem of finding the optimal weights of the
shallowest of neural networks consisting of a single Rectified Linear Unit
(ReLU). These functions are of the form $\mathbf{x}\rightarrow
\max(0,\langle\mathbf{w},\mathbf{x}\rangle)$ with $\mathbf{w}\in\mathbb{R}^d$
denoting the weight vector. We focus on a planted model where the inputs are
chosen i.i.d. from a Gaussian distribution and the labels are generated
according to a planted weight vector. We first show that mini-batch stochastic
gradient descent when suitably initialized, converges at a geometric rate to
the planted model with a number of samples that is optimal up to numerical
constants. Next we focus on a parallel implementation where in each iteration
the mini-batch gradient is calculated in a distributed manner across multiple
processors and then broadcast to a master or all other processors. To reduce
the communication cost in this setting we utilize a Quanitzed Stochastic
Gradient Scheme (QSGD) where the partial gradients are quantized. Perhaps
unexpectedly, we show that QSGD maintains the fast convergence of SGD to a
globally optimal model while significantly reducing the communication cost. We
further corroborate our numerical findings via various experiments including
distributed implementations over Amazon EC2.
Face Detection and Face Recognition In the Wild Using Off-the-Shelf Freely Available Components
Authors: Hira Ahmad
Date: 2019-01-19 00:00:00
Tasks: Face Detection, Face Recognition

This paper presents an easy and efficient face detection and face recognition
approach using free software components from the internet. Face detection and
face recognition problems have wide applications in home and office security.
Therefore this work will helpful for those searching for a free face
off-the-shelf face detection system. Using this system, faces can be detected
in uncontrolled environments. In the detection phase, every individual face is
detected and in the recognition phase the detected faces are compared with the
faces in a given data set and recognized.
Design of Real-time Semantic Segmentation Decoder for Automated Driving
Authors: Arindam Das, Saranya Kandan, Senthil Yogamani, Pavel Krizek
Date: 2019-01-19 00:00:00
Tasks: Decoder, image-classification, Image Classification, Multi-Task Learning, object-detection, Object Detection, Real-Time Semantic Segmentation, Segmentation, Semantic Segmentation

Semantic segmentation remains a computationally intensive algorithm for
embedded deployment even with the rapid growth of computation power. Thus
efficient network design is a critical aspect especially for applications like
automated driving which requires real-time performance. Recently, there has
been a lot of research on designing efficient encoders that are mostly task
agnostic. Unlike image classification and bounding box object detection tasks,
decoders are computationally expensive as well for semantic segmentation task.
In this work, we focus on efficient design of the segmentation decoder and
assume that an efficient encoder is already designed to provide shared features
for a multi-task learning system. We design a novel efficient non-bottleneck
layer and a family of decoders which fit into a small run-time budget using
VGG10 as efficient encoder. We demonstrate in our dataset that experimentation
with various design choices led to an improvement of 10\% from a baseline
performance.
Towards Physically Safe Reinforcement Learning under Supervision
Authors: Yinan Zhang, Devin Balkcom, Haoxiang Li
Date: 2019-01-19 00:00:00
Tasks: OpenAI Gym, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Reinforcement Learning

This paper addresses the question of how a previously available control
policy $\pi_s$ can be used as a supervisor to more quickly and safely train a
new learned control policy $\pi_L$ for a robot. A weighted average of the
supervisor and learned policies is used during trials, with a heavier weight
initially on the supervisor, in order to allow safe and useful physical trials
while the learned policy is still ineffective. During the process, the weight
is adjusted to favor the learned policy. As weights are adjusted, the learned
network must compensate so as to give safe and reasonable outputs under the
different weights. A pioneer network is introduced that pre-learns a policy
that performs similarly to the current learned policy under the planned next
step for new weights; this pioneer network then replaces the currently learned
network in the next set of trials. Experiments in OpenAI Gym demonstrate the
effectiveness of the proposed method.
Learning retrosynthetic planning through self-play
Authors: John S. Schreck, Connor W. Coley, Kyle J. M. Bishop
Date: 2019-01-19 00:00:00
Tasks: Deep Reinforcement Learning, Multi-step retrosynthesis, Reinforcement Learning

The problem of retrosynthetic planning can be framed as one player game, in
which the chemist (or a computer program) works backwards from a molecular
target to simpler starting materials though a series of choices regarding which
reactions to perform. This game is challenging as the combinatorial space of
possible choices is astronomical, and the value of each choice remains
uncertain until the synthesis plan is completed and its cost evaluated. Here,
we address this problem using deep reinforcement learning to identify policies
that make (near) optimal reaction choices during each step of retrosynthetic
planning. Using simulated experience or self-play, we train neural networks to
estimate the expected synthesis cost or value of any given molecule based on a
representation of its molecular structure. We show that learned policies based
on this value network outperform heuristic approaches in synthesizing
unfamiliar molecules from available starting materials using the fewest number
of reactions. We discuss how the learned policies described here can be
incorporated into existing synthesis planning tools and how they can be adapted
to changes in the synthesis cost objective or material availability.
Overfitting Mechanism and Avoidance in Deep Neural Networks
Authors: Shaeke Salman, Xiuwen Liu
Date: 2019-01-19 00:00:00
Tasks: General Classification, Object Recognition, speech-recognition, Speech Recognition

Assisted by the availability of data and high performance computing, deep
learning techniques have achieved breakthroughs and surpassed human performance
empirically in difficult tasks, including object recognition, speech
recognition, and natural language processing. As they are being used in
critical applications, understanding underlying mechanisms for their successes
and limitations is imperative. In this paper, we show that overfitting, one of
the fundamental issues in deep neural networks, is due to continuous gradient
updating and scale sensitiveness of cross entropy loss. By separating samples
into correctly and incorrectly classified ones, we show that they behave very
differently, where the loss decreases in the correct ones and increases in the
incorrect ones. Furthermore, by analyzing dynamics during training, we propose
a consensus-based classification algorithm that enables us to avoid overfitting
and significantly improve the classification accuracy especially when the
number of training samples is limited. As each trained neural network depends
on extrinsic factors such as initial values as well as training data, requiring
consensus among multiple models reduces extrinsic factors substantially; for
statistically independent models, the reduction is exponential. Compared to
ensemble algorithms, the proposed algorithm avoids overgeneralization by not
classifying ambiguous inputs. Systematic experimental results demonstrate the
effectiveness of the proposed algorithm. For example, using only 1000 training
samples from MNIST dataset, the proposed algorithm achieves 95% accuracy,
significantly higher than any of the individual models, with 90% of the test
samples classified.
Consistent Optimization for Single-Shot Object Detection
Authors: Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Jianbo Shi
Date: 2019-01-19 00:00:00
Tasks: Object, object-detection, Object Detection

We present consistent optimization for single stage object detection.
Previous works of single stage object detectors usually rely on the regular,
dense sampled anchors to generate hypothesis for the optimization of the model.
Through an examination of the behavior of the detector, we observe that the
misalignment between the optimization target and inference configurations has
hindered the performance improvement. We propose to bride this gap by
consistent optimization, which is an extension of the traditional single stage
detector's optimization strategy. Consistent optimization focuses on matching
the training hypotheses and the inference quality by utilizing of the refined
anchors during training. To evaluate its effectiveness, we conduct various
design choices based on the state-of-the-art RetinaNet detector. We demonstrate
it is the consistent optimization, not the architecture design, that yields the
performance boosts. Consistent optimization is nearly cost-free, and achieves
stable performance gains independent of the model capacities or input scales.
Specifically, utilizing consistent optimization improves RetinaNet from 39.1 AP
to 40.1 AP on COCO dataset without any bells or whistles, which surpasses the
accuracy of all existing state-of-the-art one-stage detectors when adopting
ResNet-101 as backbone. The code will be made available.
Explaining Explanations to Society
Authors: Leilani H. Gilpin, Cecilia Testart, Nathaniel Fruchter, Julius Adebayo
Date: 2019-01-19 00:00:00
Tasks: Decision Making, Explainable Artificial Intelligence (XAI)

There is a disconnect between explanatory artificial intelligence (XAI)
methods and the types of explanations that are useful for and demanded by
society (policy makers, government officials, etc.) Questions that experts in
artificial intelligence (AI) ask opaque systems provide inside explanations,
focused on debugging, reliability, and validation. These are different from
those that society will ask of these systems to build trust and confidence in
their decisions. Although explanatory AI systems can answer many questions that
experts desire, they often don't explain why they made decisions in a way that
is precise (true to the model) and understandable to humans. These outside
explanations can be used to build trust, comply with regulatory and policy
changes, and act as external validation. In this paper, we focus on XAI methods
for deep neural networks (DNNs) because of DNNs' use in decision-making and
inherent opacity. We explore the types of questions that explanatory DNN
systems can answer and discuss challenges in building explanatory systems that
provide outside explanations for societal requirements and benefit.
Synthesizing facial photometries and corresponding geometries using generative adversarial networks
Authors: Gil Shamai, Ron Slossberg, Ron Kimmel
Date: 2019-01-19 00:00:00

Artificial data synthesis is currently a well studied topic with useful
applications in data science, computer vision, graphics and many other fields.
Generating realistic data is especially challenging since human perception is
highly sensitive to non realistic appearance. In recent times, new levels of
realism have been achieved by advances in GAN training procedures and
architectures. These successful models, however, are tuned mostly for use with
regularly sampled data such as images, audio and video. Despite the successful
application of the architecture on these types of media, applying the same
tools to geometric data poses a far greater challenge. The study of geometric
deep learning is still a debated issue within the academic community as the
lack of intrinsic parametrization inherent to geometric objects prohibits the
direct use of convolutional filters, a main building block of today's machine
learning systems. In this paper we propose a new method for generating
realistic human facial geometries coupled with overlayed textures. We
circumvent the parametrization issue by imposing a global mapping from our data
to the unit rectangle. We further discuss how to design such a mapping to
control the mapping distortion and conserve area within the mapped image. By
representing geometric textures and geometries as images, we are able to use
advanced GAN methodologies to generate new geometries. We address the often
neglected topic of relation between texture and geometry and propose to use
this correlation to match between generated textures and their corresponding
geometries. We offer a new method for training GAN models on partially
corrupted data. Finally, we provide empirical evidence demonstrating our
generative model's ability to produce examples of new identities independent
from the training data while maintaining a high level of realism, two traits
that are often at odds.
MOROCO: The Moldavian and Romanian Dialectal Corpus
Authors: Andrei M. Butnaru, Radu Tudor Ionescu
Date: 2019-01-19 00:00:00
Proceeding: ACL 2019 7
Tasks: Cultural Vocal Bursts Intensity Prediction

In this work, we introduce the MOldavian and ROmanian Dialectal COrpus (MOROCO), which is freely available for download at https://github.com/butnaruandrei/MOROCO. The corpus contains 33564 samples of text (with over 10 million tokens) collected from the news domain. The samples belong to one of the following six topics: culture, finance, politics, science, sports and tech. The data set is divided into 21719 samples for training, 5921 samples for validation and another 5924 samples for testing. For each sample, we provide corresponding dialectal and category labels. This allows us to perform empirical studies on several classification tasks such as (i) binary discrimination of Moldavian versus Romanian text samples, (ii) intra-dialect multi-class categorization by topic and (iii) cross-dialect multi-class categorization by topic. We perform experiments using a shallow approach based on string kernels, as well as a novel deep approach based on character-level convolutional neural networks containing Squeeze-and-Excitation blocks. We also present and analyze the most discriminative features of our best performing model, before and after named entity removal.
Comparative Performance Analysis of Image De-noising Techniques
Authors: Vivek Kumar, Atul Samadhiya
Date: 2019-01-19 00:00:00
Tasks: Denoising, Image Denoising

Noise is an important factor which when get added to an image reduces its
quality and appearance. So in order to enhance the image qualities, it has to
be removed with preserving the textural information and structural features of
image. There are different types of noises exist who corrupt the images.
Selection of the denoising algorithm is application dependent. Hence, it is
necessary to have knowledge about the noise present in the image so as to
select the appropriate denoising algorithm. Objective of this paper is to
present brief account on types of noises, its types and different noise removal
algorithms. In the first section types of noises on the basis of their additive
and multiplicative nature are being discussed. In second section a precise
classification and analysis of the different potential image denoising
algorithm is presented. At the end of paper, a comparative study of all these
algorithms in context of performance evaluation is done and concluded with
several promising directions for future research work.
Image De-Noising For Salt and Pepper Noise by Introducing New Enhanced Filter
Authors: Vivek Kumar, Atul Samadhiya
Date: 2019-01-19 00:00:00

When an image is formed, factors such as lighting (spectra, source, and
intensity) and camera characteristics (sensor response, lenses) affect the
appearance of the image. Therefore, the prime factor that reduces the quality
of the image is noise. It hides the important details and information of
images. In order to enhance the qualities of the image, the removal of noises
become imperative and that should not at the cost of any loss of image
information. Noise removal is one of the pre-processing stages of image
processing. In this paper a new method for the enhancement of grayscale images
is introduced, when images are corrupted by fixed valued impulse noise (salt
and pepper noise). The proposed methodology ensures a better output for the low
and medium density of fixed value impulse noise as compared to the other famous
filters like Standard Median Filter (SMF), Decision Based Median Filter (DBMF)
and Modified Decision Based Median Filter (MDBMF) etc. The main objective of
the proposed method was to improve peak signal to noise ratio (PSNR), visual
perception and reduction in the blurring of the image. The proposed algorithm
replaced the noisy pixel by trimmed mean value. When previous pixel values, 0s,
and 255s are present in the particular window and all the pixel values are 0s
and 255s then the remaining noisy pixels are replaced by mean value. The
gray-scale image of mandrill and Lena were tested via the proposed method. The
experimental result shows better peak signal to noise ratio (PSNR), mean square
error values with better visual and human perception.