pwc-archive/papers-with-abstracts huggingface dataset:
Training Simplification and Model Simplification for Deep Learning: A Minimal Effort Back Propagation Method
Authors: Xu Sun, Xuancheng Ren, Shuming Ma, Bingzhen Wei, Wei Li, Jingjing Xu, Houfeng Wang, Yi Zhang
Date: 2017-11-17 00:00:00

We propose a simple yet effective technique to simplify the training and the
resulting model of neural networks. In back propagation, only a small subset of
the full gradient is computed to update the model parameters. The gradient
vectors are sparsified in such a way that only the top-k elements (in terms of
magnitude) are kept. As a result, only k rows or columns (depending on the
layout) of the weight matrix are modified, leading to a linear reduction in the
computational cost. Based on the sparsified gradients, we further simplify the
model by eliminating the rows or columns that are seldom updated, which will
reduce the computational cost both in the training and decoding, and
potentially accelerate decoding in real-world applications. Surprisingly,
experimental results demonstrate that most of time we only need to update fewer
than 5% of the weights at each back propagation pass. More interestingly, the
accuracy of the resulting models is actually improved rather than degraded, and
a detailed analysis is given. The model simplification results show that we
could adaptively simplify the model which could often be reduced by around 9x,
without any loss on accuracy or even with improved accuracy. The codes,
including the extension, are available at https://github.com/lancopku/meSimp
Collaborative Information Bottleneck
Authors: Matías Vera, Leonardo Rey Vega, Pablo Piantanida
Date: 2016-04-05 00:00:00

This paper investigates a multi-terminal source coding problem under a logarithmic loss fidelity which does not necessarily lead to an additive distortion measure. The problem is motivated by an extension of the Information Bottleneck method to a multi-source scenario where several encoders have to build cooperatively rate-limited descriptions of their sources in order to maximize information with respect to other unobserved (hidden) sources. More precisely, we study fundamental information-theoretic limits of the so-called: (i) Two-way Collaborative Information Bottleneck (TW-CIB) and (ii) the Collaborative Distributed Information Bottleneck (CDIB) problems. The TW-CIB problem consists of two distant encoders that separately observe marginal (dependent) components $X_1$ and $X_2$ and can cooperate through multiple exchanges of limited information with the aim of extracting information about hidden variables $(Y_1,Y_2)$, which can be arbitrarily dependent on $(X_1,X_2)$. On the other hand, in CDIB there are two cooperating encoders which separately observe $X_1$ and $X_2$ and a third node which can listen to the exchanges between the two encoders in order to obtain information about a hidden variable $Y$. The relevance (figure-of-merit) is measured in terms of a normalized (per-sample) multi-letter mutual information metric (log-loss fidelity) and an interesting tradeoff arises by constraining the complexity of descriptions, measured in terms of the rates needed for the exchanges between the encoders and decoders involved. Inner and outer bounds to the complexity-relevance region of these problems are derived from which optimality is characterized for several cases of interest. Our resulting theoretical complexity-relevance regions are finally evaluated for binary symmetric and Gaussian statistical models.
Classification of postoperative surgical site infections from blood measurements with missing data using recurrent neural networks
Authors: Andreas Storvik Strauman, Filippo Maria Bianchi, Karl Øyvind Mikalsen, Michael Kampffmeyer, Cristina Soguero-Ruiz, Robert Jenssen
Date: 2017-11-17 00:00:00
Tasks: General Classification, Imputation, Missing Values, Time Series, Time Series Analysis

Clinical measurements that can be represented as time series constitute an
important fraction of the electronic health records and are often both
uncertain and incomplete. Recurrent neural networks are a special class of
neural networks that are particularly suitable to process time series data but,
in their original formulation, cannot explicitly deal with missing data. In
this paper, we explore imputation strategies for handling missing values in
classifiers based on recurrent neural network (RNN) and apply a recently
proposed recurrent architecture, the Gated Recurrent Unit with Decay,
specifically designed to handle missing data. We focus on the problem of
detecting surgical site infection in patients by analyzing time series of their
blood sample measurements and we compare the results obtained with different
RNN-based classifiers.
Distributed Information-Theoretic Clustering
Authors: Georg Pichler, Pablo Piantanida, Gerald Matz
Date: 2016-02-15 00:00:00
Tasks: Clustering, Two-sample testing

We study a novel multi-terminal source coding setup motivated by the biclustering problem. Two separate encoders observe two i.i.d. sequences $X^n$ and $Y^n$, respectively. The goal is to find rate-limited encodings $f(x^n)$ and $g(z^n)$ that maximize the mutual information $I(f(X^n); g(Y^n))/n$. We discuss connections of this problem with hypothesis testing against independence, pattern recognition, and the information bottleneck method. Improving previous cardinality bounds for the inner and outer bounds allows us to thoroughly study the special case of a binary symmetric source and to quantify the gap between the inner and the outer bound in this special case. Furthermore, we investigate a multiple description (MD) extension of the Chief Operating Officer (CEO) problem with mutual information constraint. Surprisingly, this MD-CEO problem permits a tight single-letter characterization of the achievable region.
Detecting hip fractures with radiologist-level performance using deep neural networks
Authors: William Gale, Luke Oakden-Rayner, Gustavo Carneiro, Andrew P. Bradley, Lyle J. Palmer
Date: 2017-11-17 00:00:00
Tasks: Diagnostic

We developed an automated deep learning system to detect hip fractures from
frontal pelvic x-rays, an important and common radiological task. Our system
was trained on a decade of clinical x-rays (~53,000 studies) and can be applied
to clinical data, automatically excluding inappropriate and technically
unsatisfactory studies. We demonstrate diagnostic performance equivalent to a
human radiologist and an area under the ROC curve of 0.994. Translated to
clinical practice, such a system has the potential to increase the efficiency
of diagnosis, reduce the need for expensive additional testing, expand access
to expert level medical image interpretation, and improve overall patient
outcomes.
Pseudo-positive regularization for deep person re-identification
Authors: Fuqing Zhu, Xiangwei Kong, Haiyan Fu, Qi Tian
Date: 2017-11-17 00:00:00
Tasks: Data Augmentation, Diversity, Person Re-Identification

An intrinsic challenge of person re-identification (re-ID) is the annotation
difficulty. This typically means 1) few training samples per identity, and 2)
thus the lack of diversity among the training samples. Consequently, we face
high risk of over-fitting when training the convolutional neural network (CNN),
a state-of-the-art method in person re-ID. To reduce the risk of over-fitting,
this paper proposes a Pseudo Positive Regularization (PPR) method to enrich the
diversity of the training data. Specifically, unlabeled data from an
independent pedestrian database is retrieved using the target training data as
query. A small proportion of these retrieved samples are randomly selected as
the Pseudo Positive samples and added to the target training set for the
supervised CNN training. The addition of Pseudo Positive samples is therefore a
data augmentation method to reduce the risk of over-fitting during CNN
training. We implement our idea in the identification CNN models (i.e.,
CaffeNet, VGGNet-16 and ResNet-50). On CUHK03 and Market-1501 datasets,
experimental results demonstrate that the proposed method consistently improves
the baseline and yields competitive performance to the state-of-the-art person
re-ID methods.
Win Prediction in Esports: Mixed-Rank Match Prediction in Multi-player Online Battle Arena Games
Authors: Victoria Hodge, Sam Devlin, Nick Sephton, Florian Block, Anders Drachen, Peter Cowling
Date: 2017-11-17 00:00:00
Tasks: Prediction

Esports has emerged as a popular genre for players as well as spectators,
supporting a global entertainment industry. Esports analytics has evolved to
address the requirement for data-driven feedback, and is focused on
cyber-athlete evaluation, strategy and prediction. Towards the latter, previous
work has used match data from a variety of player ranks from hobbyist to
professional players. However, professional players have been shown to behave
differently than lower ranked players. Given the comparatively limited supply
of professional data, a key question is thus whether mixed-rank match datasets
can be used to create data-driven models which predict winners in professional
matches and provide a simple in-game statistic for viewers and broadcasters.
Here we show that, although there is a slightly reduced accuracy, mixed-rank
datasets can be used to predict the outcome of professional matches, with
suitably optimized configurations.
Large Neural Network Based Detection of Apnea, Bradycardia and Desaturation Events
Authors: Antoine Honoré, Veronica Siljehav, Saikat Chatterjee, Eric Herlenius
Date: 2017-11-17 00:00:00
Tasks: BIG-bench Machine Learning, Binary Classification, General Classification

Apnea, bradycardia and desaturation (ABD) events often precede
life-threatening events including sepsis in newborn babies. Here, we explore
machine learning for detection of ABD events as a binary classification
problem. We investigate the use of a large neural network to achieve a good
detection performance. To be user friendly, the chosen neural network does not
require a high level of parameter tuning. Furthermore, a limited amount of
training data is available and the training dataset is unbalanced. Comparing
with two widely used state-of-the-art machine learning algorithms, the large
neural network is found to be efficient. Even with a limited and unbalanced
training data, the large neural network provides a detection performance level
that is feasible to use in clinical care.
AI Challenger : A Large-scale Dataset for Going Deeper in Image Understanding
Authors: Jiahong Wu, He Zheng, Bo Zhao, Yixin Li, Baoming Yan, Rui Liang, Wenjia Wang, Shipei Zhou, Guosen Lin, Yanwei Fu, Yizhou Wang, Yonggang Wang
Date: 2017-11-17 00:00:00
Tasks: Attribute, General Classification, Keypoint Detection

Significant progress has been achieved in Computer Vision by leveraging
large-scale image datasets. However, large-scale datasets for complex Computer
Vision tasks beyond classification are still limited. This paper proposed a
large-scale dataset named AIC (AI Challenger) with three sub-datasets, human
keypoint detection (HKD), large-scale attribute dataset (LAD) and image Chinese
captioning (ICC). In this dataset, we annotate class labels (LAD), keypoint
coordinate (HKD), bounding box (HKD and LAD), attribute (LAD) and caption
(ICC). These rich annotations bridge the semantic gap between low-level images
and high-level concepts. The proposed dataset is an effective benchmark to
evaluate and improve different computational methods. In addition, for related
tasks, others can also use our dataset as a new resource to pre-train their
models.
Grounding Visual Explanations (Extended Abstract)
Authors: Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, Zeynep Akata
Date: 2017-11-17 00:00:00
Tasks: Attribute

Existing models which generate textual explanations enforce task relevance
through a discriminative term loss function, but such mechanisms only weakly
constrain mentioned object parts to actually be present in the image. In this
paper, a new model is proposed for generating explanations by utilizing
localized grounding of constituent phrases in generated explanations to ensure
image relevance. Specifically, we introduce a phrase-critic model to refine
(re-score/re-rank) generated candidate explanations and employ a
relative-attribute inspired ranking loss using "flipped" phrases as negative
examples for training. At test time, our phrase-critic model takes an image and
a candidate explanation as input and outputs a score indicating how well the
candidate explanation is grounded in the image.
A unified deep artificial neural network approach to partial differential equations in complex geometries
Authors: Jens Berg, Kaj Nyström
Date: 2017-11-17 00:00:00

In this paper we use deep feedforward artificial neural networks to
approximate solutions to partial differential equations in complex geometries.
We show how to modify the backpropagation algorithm to compute the partial
derivatives of the network output with respect to the space variables which is
needed to approximate the differential operator. The method is based on an
ansatz for the solution which requires nothing but feedforward neural networks
and an unconstrained gradient based optimization method such as gradient
descent or a quasi-Newton method.
  We show an example where classical mesh based methods cannot be used and
neural networks can be seen as an attractive alternative. Finally, we highlight
the benefits of deep compared to shallow neural networks and device some other
convergence enhancing techniques.
EraseReLU: A Simple Way to Ease the Training of Deep Convolution Neural Networks
Authors: Xuanyi Dong, Guoliang Kang, Kun Zhan, Yi Yang
Date: 2017-09-22 00:00:00
Tasks: Blocking, Image Classification

For most state-of-the-art architectures, Rectified Linear Unit (ReLU) becomes
a standard component accompanied with each layer. Although ReLU can ease the
network training to an extent, the character of blocking negative values may
suppress the propagation of useful information and leads to the difficulty of
optimizing very deep Convolutional Neural Networks (CNNs). Moreover, stacking
layers with nonlinear activations is hard to approximate the intrinsic linear
transformations between feature representations.
  In this paper, we investigate the effect of erasing ReLUs of certain layers
and apply it to various representative architectures following deterministic
rules. It can ease the optimization and improve the generalization performance
for very deep CNN models. We find two key factors being essential to the
performance improvement: 1) the location where ReLU should be erased inside the
basic module; 2) the proportion of basic modules to erase ReLU; We show that
erasing the last ReLU layer of all basic modules in a network usually yields
improved performance. In experiments, our approach successfully improves the
performance of various representative architectures, and we report the improved
results on SVHN, CIFAR-10/100, and ImageNet. Moreover, we achieve competitive
single-model performance on CIFAR-100 with 16.53% error rate compared to
state-of-the-art.
A Fusion-based Gender Recognition Method Using Facial Images
Authors: Benyamin Ghojogh, Saeed Bagheri Shouraki, Hoda Mohammadzade, Ensieh Iranmehr
Date: 2017-11-17 00:00:00
Tasks: Age And Gender Classification, Face Recognition, Gender Prediction

This paper proposes a fusion-based gender recognition method which uses
facial images as input. Firstly, this paper utilizes pre-processing and a
landmark detection method in order to find the important landmarks of faces.
Thereafter, four different frameworks are proposed which are inspired by
state-of-the-art gender recognition systems. The first framework extracts
features using Local Binary Pattern (LBP) and Principal Component Analysis
(PCA) and uses back propagation neural network. The second framework uses Gabor
filters, PCA, and kernel Support Vector Machine (SVM). The third framework uses
lower part of faces as input and classifies them using kernel SVM. The fourth
framework uses Linear Discriminant Analysis (LDA) in order to classify the side
outline landmarks of faces. Finally, the four decisions of frameworks are fused
using weighted voting. This paper takes advantage of both texture and
geometrical information, the two dominant types of information in facial gender
recognition. Experimental results show the power and effectiveness of the
proposed method. This method obtains recognition rate of 94% for neutral faces
of FEI face dataset, which is equal to state-of-the-art rate for this dataset.
Improving Consistency and Correctness of Sequence Inpainting using Semantically Guided Generative Adversarial Network
Authors: Avisek Lahiri, Arnav Jain, Prabir Kumar Biswas, Pabitra Mitra
Date: 2017-11-16 00:00:00
Tasks: Generative Adversarial Network, Image Inpainting

Contemporary benchmark methods for image inpainting are based on deep
generative models and specifically leverage adversarial loss for yielding
realistic reconstructions. However, these models cannot be directly applied on
image/video sequences because of an intrinsic drawback- the reconstructions
might be independently realistic, but, when visualized as a sequence, often
lacks fidelity to the original uncorrupted sequence. The fundamental reason is
that these methods try to find the best matching latent space representation
near to natural image manifold without any explicit distance based loss. In
this paper, we present a semantically conditioned Generative Adversarial
Network (GAN) for sequence inpainting. The conditional information constrains
the GAN to map a latent representation to a point in image manifold respecting
the underlying pose and semantics of the scene. To the best of our knowledge,
this is the first work which simultaneously addresses consistency and
correctness of generative model based inpainting. We show that our generative
model learns to disentangle pose and appearance information; this independence
is exploited by our model to generate highly consistent reconstructions. The
conditional information also aids the generator network in GAN to produce
sharper images compared to the original GAN formulation. This helps in
achieving more appealing inpainting performance. Though generic, our algorithm
was targeted for inpainting on faces. When applied on CelebA and Youtube Faces
datasets, the proposed method results in a significant improvement over the
current benchmark, both in terms of quantitative evaluation (Peak Signal to
Noise Ratio) and human visual scoring over diversified combinations of
resolutions and deformations.
Chinese Typeface Transformation with Hierarchical Adversarial Network
Authors: Jie Chang, Yujun Gu, Ya zhang
Date: 2017-11-17 00:00:00
Tasks: Decoder, Image Generation, Style Transfer

In this paper, we explore automated typeface generation through image style
transfer which has shown great promise in natural image generation. Existing
style transfer methods for natural images generally assume that the source and
target images share similar high-frequency features. However, this assumption
is no longer true in typeface transformation. Inspired by the recent
advancement in Generative Adversarial Networks (GANs), we propose a
Hierarchical Adversarial Network (HAN) for typeface transformation. The
proposed HAN consists of two sub-networks: a transfer network and a
hierarchical adversarial discriminator. The transfer network maps characters
from one typeface to another. A unique characteristic of typefaces is that the
same radicals may have quite different appearances in different characters even
under the same typeface. Hence, a stage-decoder is employed by the transfer
network to leverage multiple feature layers, aiming to capture both the global
and local features. The hierarchical adversarial discriminator implicitly
measures data discrepancy between the generated domain and the target domain.
To leverage the complementary discriminating capability of different feature
layers, a hierarchical structure is proposed for the discriminator. We have
experimentally demonstrated that HAN is an effective framework for typeface
transfer and characters restoration.
Multi-Objective Maximization of Monotone Submodular Functions with Cardinality Constraint
Authors: Rajan Udwani
Date: 2017-11-17 00:00:00
Proceeding: NeurIPS 2018 12

We consider the problem of multi-objective maximization of monotone submodular functions subject to cardinality constraint, often formulated as $\max_{|A|=k}\min_{i\in\{1,\dots,m\}}f_i(A)$. While it is widely known that greedy methods work well for a single objective, the problem becomes much harder with multiple objectives. In fact, Krause et al.\ (2008) showed that when the number of objectives $m$ grows as the cardinality $k$ i.e., $m=\Omega(k)$, the problem is inapproximable (unless $P=NP$). On the other hand, when $m$ is constant Chekuri et al.\ (2010) showed a randomized $(1-1/e)-\epsilon$ approximation with runtime (number of queries to function oracle) $n^{m/\epsilon^3}$. %In fact, the result of Chekuri et al.\ (2010) is for the far more general case of matroid constant. We focus on finding a fast and practical algorithm that has (asymptotic) approximation guarantees even when $m$ is super constant. We first modify the algorithm of Chekuri et al.\ (2010) to achieve a $(1-1/e)$ approximation for $m=o(\frac{k}{\log^3 k})$. This demonstrates a steep transition from constant factor approximability to inapproximability around $m=\Omega(k)$. Then using Multiplicative-Weight-Updates (MWU), we find a much faster $\tilde{O}(n/\delta^3)$ time asymptotic $(1-1/e)^2-\delta$ approximation. While the above results are all randomized, we also give a simple deterministic $(1-1/e)-\epsilon$ approximation with runtime $kn^{m/\epsilon^4}$. Finally, we run synthetic experiments using Kronecker graphs and find that our MWU inspired heuristic outperforms existing heuristics.
Action-Attending Graphic Neural Network
Authors: Chaolong Li, Zhen Cui, Wenming Zheng, Chunyan Xu, Rongrong Ji, Jian Yang
Date: 2017-11-17 00:00:00
Tasks: Action Analysis, Action Recognition, Attribute, Skeleton Based Action Recognition, Temporal Action Localization

The motion analysis of human skeletons is crucial for human action
recognition, which is one of the most active topics in computer vision. In this
paper, we propose a fully end-to-end action-attending graphic neural network
(A$^2$GNN) for skeleton-based action recognition, in which each irregular
skeleton is structured as an undirected attribute graph. To extract high-level
semantic representation from skeletons, we perform the local spectral graph
filtering on the constructed attribute graphs like the standard image
convolution operation. Considering not all joints are informative for action
analysis, we design an action-attending layer to detect those salient action
units (AUs) by adaptively weighting skeletal joints. Herein the filtering
responses are parameterized into a weighting function irrelevant to the order
of input nodes. To further encode continuous motion variations, the deep
features learnt from skeletal graphs are gathered along consecutive temporal
slices and then fed into a recurrent gated network. Finally, the spectral graph
filtering, action-attending and recurrent temporal encoding are integrated
together to jointly train for the sake of robust action recognition as well as
the intelligibility of human actions. To evaluate our A$^2$GNN, we conduct
extensive experiments on four benchmark skeleton-based action datasets,
including the large-scale challenging NTU RGB+D dataset. The experimental
results demonstrate that our network achieves the state-of-the-art
performances.