pwc-archive/papers-with-abstracts huggingface dataset:
BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks
Authors: Surat Teerapittayanon, Bradley McDanel, H. T. Kung
Date: 2017-09-06 00:00:00

Deep neural networks are state of the art methods for many learning tasks due
to their ability to extract increasingly better features at each network layer.
However, the improved performance of additional layers in a deep network comes
at the cost of added latency and energy usage in feedforward inference. As
networks continue to get deeper and larger, these costs become more prohibitive
for real-time and energy-sensitive applications. To address this issue, we
present BranchyNet, a novel deep network architecture that is augmented with
additional side branch classifiers. The architecture allows prediction results
for a large portion of test samples to exit the network early via these
branches when samples can already be inferred with high confidence. BranchyNet
exploits the observation that features learned at an early layer of a network
may often be sufficient for the classification of many data points. For more
difficult samples, which are expected less frequently, BranchyNet will use
further or all network layers to provide the best likelihood of correct
prediction. We study the BranchyNet architecture using several well-known
networks (LeNet, AlexNet, ResNet) and datasets (MNIST, CIFAR10) and show that
it can both improve accuracy and significantly reduce the inference time of the
network.
Neural Machine Translation Model with a Large Vocabulary Selected by Branching Entropy
Authors: Zi Long, Ryuichiro Kimura, Takehito Utsuro, Tomoharu Mitsuhashi, Mikio Yamamoto
Date: 2017-04-14 00:00:00
Proceeding: MTSummit 2017 9
Tasks: Machine Translation, NMT, Sentence, Translation

Neural machine translation (NMT), a new approach to machine translation, has
achieved promising results comparable to those of traditional approaches such
as statistical machine translation (SMT). Despite its recent success, NMT
cannot handle a larger vocabulary because the training complexity and decoding
complexity proportionally increase with the number of target words. This
problem becomes even more serious when translating patent documents, which
contain many technical terms that are observed infrequently. In this paper, we
propose to select phrases that contain out-of-vocabulary words using the
statistical approach of branching entropy. This allows the proposed NMT system
to be applied to a translation task of any language pair without any
language-specific knowledge about technical term identification. The selected
phrases are then replaced with tokens during training and post-translated by
the phrase translation table of SMT. Evaluation on Japanese-to-Chinese,
Chinese-to-Japanese, Japanese-to-English and English-to-Japanese patent
sentence translation proved the effectiveness of phrases selected with
branching entropy, where the proposed NMT model achieves a substantial
improvement over a baseline NMT model without our proposed technique. Moreover,
the number of translation errors of under-translation by the baseline NMT model
without our proposed technique reduces to around half by the proposed NMT
model.
Predictive Coding for Dynamic Visual Processing: Development of Functional Hierarchy in a Multiple Spatio-Temporal Scales RNN Model
Authors: Minkyu Choi, Jun Tani
Date: 2017-08-02 00:00:00

The current paper proposes a novel predictive coding type neural network
model, the predictive multiple spatio-temporal scales recurrent neural network
(P-MSTRNN). The P-MSTRNN learns to predict visually perceived human whole-body
cyclic movement patterns by exploiting multiscale spatio-temporal constraints
imposed on network dynamics by using differently sized receptive fields as well
as different time constant values for each layer. After learning, the network
becomes able to proactively imitate target movement patterns by inferring or
recognizing corresponding intentions by means of the regression of prediction
error. Results show that the network can develop a functional hierarchy by
developing a different type of dynamic structure at each layer. The paper
examines how model performance during pattern generation as well as predictive
imitation varies depending on the stage of learning. The number of limit cycle
attractors corresponding to target movement patterns increases as learning
proceeds. And, transient dynamics developing early in the learning process
successfully perform pattern generation and predictive imitation tasks. The
paper concludes that exploitation of transient dynamics facilitates successful
task performance during early learning periods.
Distributed Deep Neural Networks over the Cloud, the Edge and End Devices
Authors: Surat Teerapittayanon, Bradley McDanel, H. T. Kung
Date: 2017-09-06 00:00:00
Tasks: Distributed Computing, Object Recognition, Sensor Fusion

We propose distributed deep neural networks (DDNNs) over distributed
computing hierarchies, consisting of the cloud, the edge (fog) and end devices.
While being able to accommodate inference of a deep neural network (DNN) in the
cloud, a DDNN also allows fast and localized inference using shallow portions
of the neural network at the edge and end devices. When supported by a scalable
distributed computing hierarchy, a DDNN can scale up in neural network size and
scale out in geographical span. Due to its distributed nature, DDNNs enhance
sensor fusion, system fault tolerance and data privacy for DNN applications. In
implementing a DDNN, we map sections of a DNN onto a distributed computing
hierarchy. By jointly training these sections, we minimize communication and
resource usage for devices and maximize usefulness of extracted features which
are utilized in the cloud. The resulting system has built-in support for
automatic sensor fusion and fault tolerance. As a proof of concept, we show a
DDNN can exploit geographical diversity of sensors to improve object
recognition accuracy and reduce communication cost. In our experiment, compared
with the traditional method of offloading raw sensor data to be processed in
the cloud, DDNN locally processes most sensor data on end devices while
achieving high accuracy and is able to reduce the communication cost by a
factor of over 20x.
Deep Convolutional Neural Network for Age Estimation based on VGG-Face Model
Authors: Zakariya Qawaqneh, Arafat Abu Mallouh, Buket D. Barkana
Date: 2017-09-06 00:00:00
Tasks: Age Estimation, Face Model, Face Recognition

Automatic age estimation from real-world and unconstrained face images is
rapidly gaining importance. In our proposed work, a deep CNN model that was
trained on a database for face recognition task is used to estimate the age
information on the Adience database. This paper has three significant
contributions in this field. (1) This work proves that a CNN model, which was
trained for face recognition task, can be utilized for age estimation to
improve performance; (2) Over fitting problem can be overcome by employing a
pretrained CNN on a large database for face recognition task; (3) Not only the
number of training images and the number subjects in a training database effect
the performance of the age estimation model, but also the pre-training task of
the employed CNN determines the performance of the model.
Boosting Deep Learning Risk Prediction with Generative Adversarial Networks for Electronic Health Records
Authors: Zhengping Che, Yu Cheng, Shuangfei Zhai, Zhaonan Sun, Yan Liu
Date: 2017-09-06 00:00:00
Tasks: Generative Adversarial Network, Prediction

The rapid growth of Electronic Health Records (EHRs), as well as the
accompanied opportunities in Data-Driven Healthcare (DDH), has been attracting
widespread interests and attentions. Recent progress in the design and
applications of deep learning methods has shown promising results and is
forcing massive changes in healthcare academia and industry, but most of these
methods rely on massive labeled data. In this work, we propose a general deep
learning framework which is able to boost risk prediction performance with
limited EHR data. Our model takes a modified generative adversarial network
namely ehrGAN, which can provide plausible labeled EHR data by mimicking real
patient records, to augment the training dataset in a semi-supervised learning
manner. We use this generative model together with a convolutional neural
network (CNN) based prediction model to improve the onset prediction
performance. Experiments on two real healthcare datasets demonstrate that our
proposed framework produces realistic data samples and achieves significant
improvements on classification tasks with the generated data over several
stat-of-the-art baselines.
Using Cross-Model EgoSupervision to Learn Cooperative Basketball Intention
Authors: Gedas Bertasius, Jianbo Shi
Date: 2017-09-05 00:00:00
Tasks: Pose Estimation

We present a first-person method for cooperative basketball intention
prediction: we predict with whom the camera wearer will cooperate in the near
future from unlabeled first-person images. This is a challenging task that
requires inferring the camera wearer's visual attention, and decoding the
social cues of other players. Our key observation is that a first-person view
provides strong cues to infer the camera wearer's momentary visual attention,
and his/her intentions. We exploit this observation by proposing a new
cross-model EgoSupervision learning scheme that allows us to predict with whom
the camera wearer will cooperate in the near future, without using manually
labeled intention labels. Our cross-model EgoSupervision operates by
transforming the outputs of a pretrained pose-estimation network, into pseudo
ground truth labels, which are then used as a supervisory signal to train a new
network for a cooperative intention task. We evaluate our method, and show that
it achieves similar or even better accuracy than the fully supervised methods
do.
Exploring and Exploiting Diversity for Image Segmentation
Authors: Payman Yadollahpour
Date: 2017-09-05 00:00:00
Tasks: Diversity, Image Segmentation, Re-Ranking, Segmentation, Semantic Segmentation

Semantic image segmentation is an important computer vision task that is
difficult because it consists of both recognition and segmentation. The task is
often cast as a structured output problem on an exponentially large
output-space, which is typically modeled by a discrete probabilistic model. The
best segmentation is found by inferring the Maximum a-Posteriori (MAP) solution
over the output distribution defined by the model. Due to limitations in
optimization, the model cannot be arbitrarily complex. This leads to a
trade-off: devise a more accurate model that incorporates rich high-order
interactions between image elements at the cost of inaccurate and possibly
intractable optimization OR leverage a tractable model which produces less
accurate MAP solutions but may contain high quality solutions as other modes of
its output distribution.
  This thesis investigates the latter and presents a two stage approach to
semantic segmentation. In the first stage a tractable segmentation model
outputs a set of high probability segmentations from the underlying
distribution that are not just minor perturbations of each other. Critically
the output of this stage is a diverse set of plausible solutions and not just a
single one. In the second stage, a discriminatively trained re-ranking model
selects the best segmentation from this set. The re-ranking stage can use much
more complex features than what could be tractably used in the segmentation
model, allowing a better exploration of the solution space than simply
returning the MAP solution. The formulation is agnostic to the underlying
segmentation model (e.g. CRF, CNN, etc.) and optimization algorithm, which
makes it applicable to a wide range of models and inference methods. Evaluation
of the approach on a number of semantic image segmentation benchmark datasets
highlight its superiority over inferring the MAP solution.
Human Pose Forecasting via Deep Markov Models
Authors: Sam Toyer, Anoop Cherian, Tengda Han, Stephen Gould
Date: 2017-07-24 00:00:00
Tasks: Autonomous Driving, Human Pose Forecasting

Human pose forecasting is an important problem in computer vision with
applications to human-robot interaction, visual surveillance, and autonomous
driving. Usually, forecasting algorithms use 3D skeleton sequences and are
trained to forecast for a few milliseconds into the future. Long-range
forecasting is challenging due to the difficulty of estimating how long a
person continues an activity. To this end, our contributions are threefold: (i)
we propose a generative framework for poses using variational autoencoders
based on Deep Markov Models (DMMs); (ii) we evaluate our pose forecasts using a
pose-based action classifier, which we argue better reflects the subjective
quality of pose forecasts than distance in coordinate space; (iii) last, for
evaluation of the new model, we introduce a 480,000-frame video dataset called
Ikea Furniture Assembly (Ikea FA), which depicts humans repeatedly assembling
and disassembling furniture. We demonstrate promising results for our approach
on both Ikea FA and the existing NTU RGB+D dataset.
Deep Learning Techniques for Music Generation -- A Survey
Authors: Jean-Pierre Briot, Gaëtan Hadjeres, François-David Pachet
Date: 2017-09-05 00:00:00
Tasks: Deep Learning, Music Generation, Survey

This paper is a survey and an analysis of different ways of using deep learning (deep artificial neural networks) to generate musical content. We propose a methodology based on five dimensions for our analysis: Objective - What musical content is to be generated? Examples are: melody, polyphony, accompaniment or counterpoint. - For what destination and for what use? To be performed by a human(s) (in the case of a musical score), or by a machine (in the case of an audio file). Representation - What are the concepts to be manipulated? Examples are: waveform, spectrogram, note, chord, meter and beat. - What format is to be used? Examples are: MIDI, piano roll or text. - How will the representation be encoded? Examples are: scalar, one-hot or many-hot. Architecture - What type(s) of deep neural network is (are) to be used? Examples are: feedforward network, recurrent network, autoencoder or generative adversarial networks. Challenge - What are the limitations and open challenges? Examples are: variability, interactivity and creativity. Strategy - How do we model and control the process of generation? Examples are: single-step feedforward, iterative feedforward, sampling or input manipulation. For each dimension, we conduct a comparative analysis of various models and techniques and we propose some tentative multidimensional typology. This typology is bottom-up, based on the analysis of many existing deep-learning based systems for music generation selected from the relevant literature. These systems are described and are used to exemplify the various choices of objective, representation, architecture, challenge and strategy. The last section includes some discussion and some prospects.
PageNet: Page Boundary Extraction in Historical Handwritten Documents
Authors: Chris Tensmeyer, Brian Davis, Curtis Wigington, Iain Lee, Bill Barrett
Date: 2017-09-05 00:00:00

When digitizing a document into an image, it is common to include a
surrounding border region to visually indicate that the entire document is
present in the image. However, this border should be removed prior to automated
processing. In this work, we present a deep learning based system, PageNet,
which identifies the main page region in an image in order to segment content
from both textual and non-textual border noise. In PageNet, a Fully
Convolutional Network obtains a pixel-wise segmentation which is post-processed
into the output quadrilateral region. We evaluate PageNet on 4 collections of
historical handwritten documents and obtain over 94% mean intersection over
union on all datasets and approach human performance on 2 of these collections.
Additionally, we show that PageNet can segment documents that are overlayed on
top of other documents.
Machine Learning and Social Robotics for Detecting Early Signs of Dementia
Authors: Patrik Jonell, Joseph Mendelson, Thomas Storskog, Goran Hagman, Per Ostberg, Iolanda Leite, Taras Kucherenko, Olga Mikheeva, Ulrika Akenine, Vesna Jelic, Alina Solomon, Jonas Beskow, Joakim Gustafson, Miia Kivipelto, Hedvig Kjellstrom
Date: 2017-09-05 00:00:00
Tasks: BIG-bench Machine Learning

This paper presents the EACare project, an ambitious multi-disciplinary
collaboration with the aim to develop an embodied system, capable of carrying
out neuropsychological tests to detect early signs of dementia, e.g., due to
Alzheimer's disease. The system will use methods from Machine Learning and
Social Robotics, and be trained with examples of recorded clinician-patient
interactions. The interaction will be developed using a participatory design
approach. We describe the scope and method of the project, and report on a
first Wizard of Oz prototype.
A second order primal-dual method for nonsmooth convex composite optimization
Authors: Neil K. Dhingra, Sei Zhen Khong, Mihailo R. Jovanović
Date: 2017-09-05 00:00:00
Tasks: Model Predictive Control

We develop a second order primal-dual method for optimization problems in which the objective function is given by the sum of a strongly convex twice differentiable term and a possibly nondifferentiable convex regularizer. After introducing an auxiliary variable, we utilize the proximal operator of the nonsmooth regularizer to transform the associated augmented Lagrangian into a function that is once, but not twice, continuously differentiable. The saddle point of this function corresponds to the solution of the original optimization problem. We employ a generalization of the Hessian to define second order updates on this function and prove global exponential stability of the corresponding differential inclusion. Furthermore, we develop a globally convergent customized algorithm that utilizes the primal-dual augmented Lagrangian as a merit function. We show that the search direction can be computed efficiently and prove quadratic/superlinear asymptotic convergence. We use the $\ell_1$-regularized model predictive control problem and the problem of designing a distributed controller for a spatially-invariant system to demonstrate the merits and the effectiveness of our method.
Dynamic Multiscale Tree Learning Using Ensemble Strong Classifiers for Multi-label Segmentation of Medical Images with Lesions
Authors: Samya Amiri, Mohamed Ali Mahjoub, Islem Rekik
Date: 2017-09-05 00:00:00
Tasks: General Classification, Image Segmentation, Segmentation, Semantic Segmentation, Superpixels

We introduce a dynamic multiscale tree (DMT) architecture that learns how to
leverage the strengths of different existing classifiers for supervised
multi-label image segmentation. Unlike previous works that simply aggregate or
cascade classifiers for addressing image segmentation and labeling tasks, we
propose to embed strong classifiers into a tree structure that allows
bi-directional flow of information between its classifier nodes to gradually
improve their performances. Our DMT is a generic classification model that
inherently embeds different cascades of classifiers while enhancing learning
transfer between them to boost up their classification accuracies.
Specifically, each node in our DMT can nest a Structured Random Forest (SRF)
classifier or a Bayesian Network (BN) classifier. The proposed SRF-BN DMT
architecture has several appealing properties. First, while SRF operates at a
patch-level (regular image region), BN operates at the super-pixel level
(irregular image region), thereby enabling the DMT to integrate multi-level
image knowledge in the learning process. Second, although BN is powerful in
modeling dependencies between image elements (superpixels, edges) and their
features, the learning of its structure and parameters is challenging. On the
other hand, SRF may fail to accurately detect very irregular object boundaries.
The proposed DMT robustly overcomes these limitations for both classifiers
through the ascending and descending flow of contextual information between
each parent node and its children nodes. Third, we train DMT using different
scales, where we progressively decrease the patch and superpixel sizes as we go
deeper along the tree edges nearing its leaf nodes. Last, DMT demonstrates its
outperformance in comparison to several state-of-the-art segmentation methods
for multi-labeling of brain images with gliomas.
Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks
Authors: Yipeng Hu, Eli Gibson, Tom Vercauteren, Hashim U. Ahmed, Mark Emberton, Caroline M. Moore, J. Alison Noble, Dean C. Barratt
Date: 2017-09-05 00:00:00
Tasks: Diagnostic, Specificity

In this paper, we describe how a patient-specific, ultrasound-probe-induced
prostate motion model can be directly generated from a single preoperative MR
image. Our motion model allows for sampling from the conditional distribution
of dense displacement fields, is encoded by a generative neural network
conditioned on a medical image, and accepts random noise as additional input.
The generative network is trained by a minimax optimisation with a second
discriminative neural network, tasked to distinguish generated samples from
training motion data. In this work, we propose that 1) jointly optimising a
third conditioning neural network that pre-processes the input image, can
effectively extract patient-specific features for conditioning; and 2)
combining multiple generative models trained separately with heuristically
pre-disjointed training data sets can adequately mitigate the problem of mode
collapse. Trained with diagnostic T2-weighted MR images from 143 real patients
and 73,216 3D dense displacement fields from finite element simulations of
intraoperative prostate motion due to transrectal ultrasound probe pressure,
the proposed models produced physically-plausible patient-specific motion of
prostate glands. The ability to capture biomechanically simulated motion was
evaluated using two errors representing generalisability and specificity of the
model. The median values, calculated from a 10-fold cross-validation, were
2.8+/-0.3 mm and 1.7+/-0.1 mm, respectively. We conclude that the introduced
approach demonstrates the feasibility of applying state-of-the-art machine
learning algorithms to generate organ motion models from patient images, and
shows significant promise for future research.
SegICP: Integrated Deep Semantic Segmentation and Pose Estimation
Authors: Jay M. Wong, Vincent Kee, Tiffany Le, Syler Wagner, Gian-Luca Mariottini, Abraham Schneider, Lei Hamilton, Rahul Chipalkatty, Mitchell Hebert, David M. S. Johnson, Jimmy Wu, Bolei Zhou, Antonio Torralba
Date: 2017-03-05 00:00:00
Tasks: Object Recognition, Point Cloud Registration, Pose Estimation, Position, Semantic Segmentation

Recent robotic manipulation competitions have highlighted that sophisticated
robots still struggle to achieve fast and reliable perception of task-relevant
objects in complex, realistic scenarios. To improve these systems' perceptive
speed and robustness, we present SegICP, a novel integrated solution to object
recognition and pose estimation. SegICP couples convolutional neural networks
and multi-hypothesis point cloud registration to achieve both robust pixel-wise
semantic segmentation as well as accurate and real-time 6-DOF pose estimation
for relevant objects. Our architecture achieves 1cm position error and
<5^\circ$ angle error in real time without an initial seed. We evaluate and
benchmark SegICP against an annotated dataset generated by motion capture.