pwc-archive/papers-with-abstracts huggingface dataset:
Multimodal Deep Neural Networks using Both Engineered and Learned Representations for Biodegradability Prediction
Authors: Garrett B. Goh, Khushmeen Sakloth, Charles Siegel, Abhinav Vishnu, Jim Pfaendtner
Date: 2018-08-13 00:00:00
Tasks: Feature Engineering, Representation Learning

Deep learning algorithms excel at extracting patterns from raw data, and with
large datasets, they have been very successful in computer vision and natural
language applications. However, in other domains, large datasets on which to
learn representations from may not exist. In this work, we develop a novel
multimodal CNN-MLP neural network architecture that utilizes both
domain-specific feature engineering as well as learned representations from raw
data. We illustrate the effectiveness of such network designs in the chemical
sciences, for predicting biodegradability. DeepBioD, a multimodal CNN-MLP
network is more accurate than either standalone network designs, and achieves
an error classification rate of 0.125 that is 27% lower than the current
state-of-the-art. Thus, our work indicates that combining traditional feature
engineering with representation learning can be effective, particularly in
situations where labeled data is limited.
Murmur Detection Using Parallel Recurrent & Convolutional Neural Networks
Authors: Shahnawaz Alam, Rohan Banerjee, Soma Bandyopadhyay
Date: 2018-08-13 00:00:00
Tasks: General Classification, Sensitivity, Specificity

In this article, we propose a novel technique for classification of the
Murmurs in heart sound. We introduce a novel deep neural network architecture
using parallel combination of the Recurrent Neural Network (RNN) based
Bidirectional Long Short-Term Memory (BiLSTM) & Convolutional Neural Network
(CNN) to learn visual and time-dependent characteristics of Murmur in PCG
waveform. Set of acoustic features are presented to our proposed deep neural
network to discriminate between Normal and Murmur class. The proposed method
was evaluated on a large dataset using 5-fold cross-validation, resulting in a
sensitivity and specificity of 96 +- 0.6 % , 100 +- 0 % respectively and F1
Score of 98 +- 0.3 %.
RedSync : Reducing Synchronization Traffic for Distributed Deep Learning
Authors: Jiarui Fang, Haohuan Fu, Guangwen Yang, Cho-Jui Hsieh
Date: 2018-08-13 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Learning, GPU, image-classification, Image Classification, Language Modeling, Language Modelling

Data parallelism has become a dominant method to scale Deep Neural Network (DNN) training across multiple nodes. Since synchronizing a large number of gradients of the local model can be a bottleneck for large-scale distributed training, compressing communication data has gained widespread attention recently. Among several recent proposed compression algorithms, Residual Gradient Compression (RGC) is one of the most successful approaches---it can significantly compress the transmitting message size (0.1\% of the gradient size) of each node and still achieve correct accuracy and the same convergence speed. However, the literature on compressing deep networks focuses almost exclusively on achieving good theoretical compression rate, while the efficiency of RGC in real distributed implementation has been less investigated. In this paper, we develop an RGC-based system that is able to reduce the end-to-end training time on real-world multi-GPU systems. Our proposed design called RedSync, which introduces a set of optimizations to reduce communication bandwidth requirement while introducing limited overhead. We evaluate the performance of RedSync on two different multiple GPU platforms, including 128 GPUs of a supercomputer and an 8-GPU server. Our test cases include image classification tasks on Cifar10 and ImageNet, and language modeling tasks on Penn Treebank and Wiki2 datasets. For DNNs featured with high communication to computation ratio, which have long been considered with poor scalability, RedSync brings significant performance improvements.
Cross-Cultural and Cultural-Specific Production and Perception of Facial Expressions of Emotion in the Wild
Authors: Ramprakash Srinivasan, Aleix M. Martinez
Date: 2018-08-13 00:00:00

Automatic recognition of emotion from facial expressions is an intense area
of research, with a potentially long list of important application. Yet, the
study of emotion requires knowing which facial expressions are used within and
across cultures in the wild, not in controlled lab conditions; but such studies
do not exist. Which and how many cross-cultural and cultural-specific facial
expressions do people commonly use? And, what affect variables does each
expression communicate to observers? If we are to design technology that
understands the emotion of users, we need answers to these two fundamental
questions. In this paper, we present the first large-scale study of the
production and visual perception of facial expressions of emotion in the wild.
We find that of the 16,384 possible facial configurations that people can
theoretically produce, only 35 are successfully used to transmit emotive
information across cultures, and only 8 within a smaller number of cultures.
Crucially, we find that visual analysis of cross-cultural expressions yields
consistent perception of emotion categories and valence, but not arousal. In
contrast, visual analysis of cultural-specific expressions yields consistent
perception of valence and arousal, but not of emotion categories. Additionally,
we find that the number of expressions used to communicate each emotion is also
different, e.g., 17 expressions transmit happiness, but only 1 is used to
convey disgust.
A Survey on Methods and Theories of Quantized Neural Networks
Authors: Yunhui Guo
Date: 2018-08-13 00:00:00
Tasks: Quantization, speech-recognition, Speech Recognition, Survey

Deep neural networks are the state-of-the-art methods for many real-world
tasks, such as computer vision, natural language processing and speech
recognition. For all its popularity, deep neural networks are also criticized
for consuming a lot of memory and draining battery life of devices during
training and inference. This makes it hard to deploy these models on mobile or
embedded devices which have tight resource constraints. Quantization is
recognized as one of the most effective approaches to satisfy the extreme
memory requirements that deep neural network models demand. Instead of adopting
32-bit floating point format to represent weights, quantized representations
store weights using more compact formats such as integers or even binary
numbers. Despite a possible degradation in predictive performance, quantization
provides a potential solution to greatly reduce the model size and the energy
consumption. In this survey, we give a thorough review of different aspects of
quantized neural networks. Current challenges and trends of quantized neural
networks are also discussed.
What is wrong with style transfer for texts?
Authors: Alexey Tikhonov, Ivan P. Yamshchikov
Date: 2018-08-13 00:00:00
Tasks: BIG-bench Machine Learning, Style Transfer

A number of recent machine learning papers work with an automated style
transfer for texts and, counter to intuition, demonstrate that there is no
consensus formulation of this NLP task. Different researchers propose different
algorithms, datasets and target metrics to address it. This short opinion paper
aims to discuss possible formalization of this NLP task in anticipation of a
further growing interest to it.
D-PAGE: Diverse Paraphrase Generation
Authors: Qiongkai Xu, Juyan Zhang, Lizhen Qu, Lexing Xie, Richard Nock
Date: 2018-08-13 00:00:00
Tasks: Diversity, Machine Translation, NMT, Paraphrase Generation, Translation

In this paper, we investigate the diversity aspect of paraphrase generation.
Prior deep learning models employ either decoding methods or add random input
noise for varying outputs. We propose a simple method Diverse Paraphrase
Generation (D-PAGE), which extends neural machine translation (NMT) models to
support the generation of diverse paraphrases with implicit rewriting patterns.
Our experimental results on two real-world benchmark datasets demonstrate that
our model generates at least one order of magnitude more diverse outputs than
the baselines in terms of a new evaluation metric Jeffrey's Divergence. We have
also conducted extensive experiments to understand various properties of our
model with a focus on diversity.
Pixel Objectness: Learning to Segment Generic Objects Automatically in Images and Videos
Authors: Bo Xiong, Suyog Dutt Jain, Kristen Grauman
Date: 2018-08-11 00:00:00
Tasks: Foreground Segmentation, Image Retargeting, Image Retrieval, Image Segmentation, Object, Retrieval, Segmentation, Semantic Segmentation, Structured Prediction, Video Segmentation, Video Semantic Segmentation

We propose an end-to-end learning framework for segmenting generic objects in
both images and videos. Given a novel image or video, our approach produces a
pixel-level mask for all "object-like" regions---even for object categories
never seen during training. We formulate the task as a structured prediction
problem of assigning an object/background label to each pixel, implemented
using a deep fully convolutional network. When applied to a video, our model
further incorporates a motion stream, and the network learns to combine both
appearance and motion and attempts to extract all prominent objects whether
they are moving or not. Beyond the core model, a second contribution of our
approach is how it leverages varying strengths of training annotations.
Pixel-level annotations are quite difficult to obtain, yet crucial for training
a deep network approach for segmentation. Thus we propose ways to exploit
weakly labeled data for learning dense foreground segmentation. For images, we
show the value in mixing object category examples with image-level labels
together with relatively few images with boundary-level annotations. For video,
we show how to bootstrap weakly annotated videos together with the network
trained for image segmentation. Through experiments on multiple challenging
image and video segmentation benchmarks, our method offers consistently strong
results and improves the state-of-the-art for fully automatic segmentation of
generic (unseen) objects. In addition, we demonstrate how our approach benefits
image retrieval and image retargeting, both of which flourish when given our
high-quality foreground maps. Code, models, and videos are
at:http://vision.cs.utexas.edu/projects/pixelobjectness/
A Domain Guided CNN Architecture for Predicting Age from Structural Brain Images
Authors: Pascal Sturmfels, Saige Rutherford, Mike Angstadt, Mark Peterson, Chandra Sripada, Jenna Wiens
Date: 2018-08-11 00:00:00
Tasks: Object Recognition

Given the wide success of convolutional neural networks (CNNs) applied to
natural images, researchers have begun to apply them to neuroimaging data. To
date, however, exploration of novel CNN architectures tailored to neuroimaging
data has been limited. Several recent works fail to leverage the 3D structure
of the brain, instead treating the brain as a set of independent 2D slices.
Approaches that do utilize 3D convolutions rely on architectures developed for
object recognition tasks in natural 2D images. Such architectures make
assumptions about the input that may not hold for neuroimaging. For example,
existing architectures assume that patterns in the brain exhibit translation
invariance. However, a pattern in the brain may have different meaning
depending on where in the brain it is located. There is a need to explore novel
architectures that are tailored to brain images. We present two simple
modifications to existing CNN architectures based on brain image structure.
Applied to the task of brain age prediction, our network achieves a mean
absolute error (MAE) of 1.4 years and trains 30% faster than a CNN baseline
that achieves a MAE of 1.6 years. Our results suggest that lessons learned from
developing models on natural images may not directly transfer to neuroimaging
tasks. Instead, there remains a large space of unexplored questions regarding
model development in this area, whose answers may differ from conventional
wisdom.
Community Regularization of Visually-Grounded Dialog
Authors: Akshat Agarwal, Swaminathan Gurumurthy, Vasu  Sharma, Mike Lewis, Katia Sycara
Date: 2018-08-10 00:00:00

The task of conducting visually grounded dialog involves learning
goal-oriented cooperative dialog between autonomous agents who exchange
information about a scene through several rounds of questions and answers in
natural language. We posit that requiring artificial agents to adhere to the
rules of human language, while also requiring them to maximize information
exchange through dialog is an ill-posed problem. We observe that humans do not
stray from a common language because they are social creatures who live in
communities, and have to communicate with many people everyday, so it is far
easier to stick to a common language even at the cost of some efficiency loss.
Using this as inspiration, we propose and evaluate a multi-agent
community-based dialog framework where each agent interacts with, and learns
from, multiple agents, and show that this community-enforced regularization
results in more relevant and coherent dialog (as judged by human evaluators)
without sacrificing task performance (as judged by quantitative metrics).
Image Registration and Predictive Modeling: Learning the Metric on the Space of Diffeomorphisms
Authors: Ayagoz Mussabayeva, Alexey Kroshnin, Anvar Kurmukov, Yulia Dodonova, Li Shen, Shan Cong, Lei Wang, Boris A. Gutman
Date: 2018-08-10 00:00:00
Tasks: General Classification, Image Registration

We present a method for metric optimization in the Large Deformation
Diffeomorphic Metric Mapping (LDDMM) framework, by treating the induced
Riemannian metric on the space of diffeomorphisms as a kernel in a machine
learning context. For simplicity, we choose the kernel Fischer Linear
Discriminant Analysis (KLDA) as the framework. Optimizing the kernel parameters
in an Expectation-Maximization framework, we define model fidelity via the
hinge loss of the decision function. The resulting algorithm optimizes the
parameters of the LDDMM norm-inducing differential operator as a solution to a
group-wise registration and classification problem. In practice, this may lead
to a biology-aware registration, focusing its attention on the predictive task
at hand such as identifying the effects of disease. We first tested our
algorithm on a synthetic dataset, showing that our parameter selection improves
registration quality and classification accuracy. We then tested the algorithm
on 3D subcortical shapes from the Schizophrenia cohort Schizconnect. Our
Schizpohrenia-Control predictive model showed significant improvement in ROC
AUC compared to baseline parameters.
Road Segmentation Using CNN and Distributed LSTM
Authors: Yecheng Lyu, Lin Bai, Xinming Huang
Date: 2018-08-10 00:00:00
Tasks: Road Segmentation, Segmentation

In automated driving systems (ADS) and advanced driver-assistance systems
(ADAS), an efficient road segmentation is necessary to perceive the drivable
region and build an occupancy map for path planning. The existing algorithms
implement gigantic convolutional neural networks (CNNs) that are
computationally expensive and time consuming. In this paper, we introduced
distributed LSTM, a neural network widely used in audio and video processing,
to process rows and columns in images and feature maps. We then propose a new
network combining the convolutional and distributed LSTM layers to solve the
road segmentation problem. In the end, the network is trained and tested in
KITTI road benchmark. The result shows that the combined structure enhances the
feature extraction and processing but takes less processing time than pure CNN
structure.
Out of the Black Box: Properties of deep neural networks and their applications
Authors: Nizar Ouarti, David Carmona
Date: 2018-08-10 00:00:00

Deep neural networks are powerful machine learning approaches that have
exhibited excellent results on many classification tasks. However, they are
considered as black boxes and some of their properties remain to be formalized.
In the context of image recognition, it is still an arduous task to understand
why an image is recognized or not. In this study, we formalize some properties
shared by eight state-of-the-art deep neural networks in order to grasp the
principles allowing a given deep neural network to classify an image. Our
results, tested on these eight networks, show that an image can be sub-divided
into several regions (patches) responding at different degrees of probability
(local property). With the same patch, some locations in the image can answer
two (or three) orders of magnitude higher than other locations (spatial
property). Some locations are activators and others inhibitors
(activation-inhibition property). The repetition of the same patch can increase
(or decrease) the probability of recognition of an object (cumulative
property). Furthermore, we propose a new approach called Deepception that
exploits these properties to deceive a deep neural network. We obtain for the
VGG-VDD-19 neural network a fooling ratio of 88\%. Thanks to our
"Psychophysics" approach, no prior knowledge on the networks architectures is
required.
A Hassle-Free Machine Learning Method for Cohort Selection of Clinical Trials
Authors: Liu Man
Date: 2018-08-10 00:00:00
Tasks: BIG-bench Machine Learning, Clinical Knowledge, General Classification, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Sentence, text-classification, Text Classification

Traditional text classification techniques in clinical domain have heavily
relied on the manually extracted textual cues. This paper proposes a generally
supervised machine learning method that is equally hassle-free and does not use
clinical knowledge. The employed methods were simple to implement, fast to run
and yet effective. This paper proposes a novel named entity recognition (NER)
based an ensemble system capable of learning the keyword features in the
document. Instead of merely considering the whole sentence/paragraph for
analysis, the NER based keyword features can stress the important clinic
relevant phases more. In addition, to capture the semantic information in the
documents, the FastText features originating from the document level FastText
classification results are exploited.
Mining Threat Intelligence about Open-Source Projects and Libraries from Code Repository Issues and Bug Reports
Authors: Lorenzo Neil, Sudip Mittal, Anupam Joshi
Date: 2018-08-09 00:00:00

Open-Source Projects and Libraries are being used in software development
while also bearing multiple security vulnerabilities. This use of third party
ecosystem creates a new kind of attack surface for a product in development. An
intelligent attacker can attack a product by exploiting one of the
vulnerabilities present in linked projects and libraries.
  In this paper, we mine threat intelligence about open source projects and
libraries from bugs and issues reported on public code repositories. We also
track library and project dependencies for installed software on a client
machine. We represent and store this threat intelligence, along with the
software dependencies in a security knowledge graph. Security analysts and
developers can then query and receive alerts from the knowledge graph if any
threat intelligence is found about linked libraries and projects, utilized in
their products.
Character-Level Language Modeling with Deeper Self-Attention
Authors: Rami Al-Rfou, Dokook Choe, Noah Constant, Mandy Guo, Llion Jones
Date: 2018-08-09 00:00:00
Tasks: Language Modeling, Language Modelling

LSTMs and other RNN variants have shown strong performance on character-level
language modeling. These models are typically trained using truncated
backpropagation through time, and it is common to assume that their success
stems from their ability to remember long-term contexts. In this paper, we show
that a deep (64-layer) transformer model with fixed context outperforms RNN
variants by a large margin, achieving state of the art on two popular
benchmarks: 1.13 bits per character on text8 and 1.06 on enwik8. To get good
results at this depth, we show that it is important to add auxiliary losses,
both at intermediate network layers and intermediate sequence positions.
Deep Morphing: Detecting bone structures in fluoroscopic X-ray images with prior knowledge
Authors: Aaron Pries, Peter J. Schreier, Artur Lamm, Stefan Pede, Jürgen Schmidt
Date: 2018-08-09 00:00:00

We propose approaches based on deep learning to localize objects in images
when only a small training dataset is available and the images have low
quality. That applies to many problems in medical image processing, and in
particular to the analysis of fluoroscopic (low-dose) X-ray images, where the
images have low contrast. We solve the problem by incorporating high-level
information about the objects, which could be a simple geometrical model, like
a circular outline, or a more complex statistical model. A simple geometrical
representation can sufficiently describe some objects and only requires minimal
labeling. Statistical shape models can be used to represent more complex
objects. We propose computationally efficient two-stage approaches, which we
call deep morphing, for both representations by fitting the representation to
the output of a deep segmentation network.
Deep Learning Based Natural Language Processing for End to End Speech Translation
Authors: Sarvesh Patil
Date: 2018-08-09 00:00:00
Tasks: Speech-to-Text, Translation

Deep Learning methods employ multiple processing layers to learn hierarchial
representations of data. They have already been deployed in a humongous number
of applications and have produced state-of-the-art results. Recently with the
growth in processing power of computers to be able to do high dimensional
tensor calculations, Natural Language Processing (NLP) applications have been
given a significant boost in terms of efficiency as well as accuracy. In this
paper, we will take a look at various signal processing techniques and then
application of them to produce a speech-to-text system using Deep Recurrent
Neural Networks.
Neural Machine Translation Inspired Binary Code Similarity Comparison beyond Function Pairs
Authors: Fei Zuo, Xiaopeng Li, Patrick Young, Lannan Luo, Qiang Zeng, Zhexin Zhang
Date: 2018-08-08 00:00:00
Tasks: Machine Translation, Translation

Binary code analysis allows analyzing binary code without having access to
the corresponding source code. A binary, after disassembly, is expressed in an
assembly language. This inspires us to approach binary analysis by leveraging
ideas and techniques from Natural Language Processing (NLP), a rich area
focused on processing text of various natural languages. We notice that binary
code analysis and NLP share a lot of analogical topics, such as semantics
extraction, summarization, and classification. This work utilizes these ideas
to address two important code similarity comparison problems. (I) Given a pair
of basic blocks for different instruction set architectures (ISAs), determining
whether their semantics is similar or not; and (II) given a piece of code of
interest, determining if it is contained in another piece of assembly code for
a different ISA. The solutions to these two problems have many applications,
such as cross-architecture vulnerability discovery and code plagiarism
detection. We implement a prototype system INNEREYE and perform a comprehensive
evaluation. A comparison between our approach and existing approaches to
Problem I shows that our system outperforms them in terms of accuracy,
efficiency and scalability. And the case studies utilizing the system
demonstrate that our solution to Problem II is effective. Moreover, this
research showcases how to apply ideas and techniques from NLP to large-scale
binary code analysis.