pwc-archive/papers-with-abstracts huggingface dataset:
Deformable Part Networks
Authors: Ziming Zhang, Rongmei Lin, Alan Sullivan
Date: 2018-05-22 00:00:00
Tasks: Object Recognition

In this paper we propose novel Deformable Part Networks (DPNs) to learn {\em
pose-invariant} representations for 2D object recognition. In contrast to the
state-of-the-art pose-aware networks such as CapsNet \cite{sabour2017dynamic}
and STN \cite{jaderberg2015spatial}, DPNs can be naturally {\em interpreted} as
an efficient solver for a challenging detection problem, namely Localized
Deformable Part Models (LDPMs) where localization is introduced to DPMs as
another latent variable for searching for the best poses of objects over all
pixels and (predefined) scales. In particular we construct DPNs as sequences of
such LDPM units to model the semantic and spatial relations among the
deformable parts as hierarchical composition and spatial parsing trees.
Empirically our 17-layer DPN can outperform both CapsNets and STNs
significantly on affNIST \cite{sabour2017dynamic}, for instance, by 19.19\% and
12.75\%, respectively, with better generalization and better tolerance to
affine transformations.
Equivalence of Equilibrium Propagation and Recurrent Backpropagation
Authors: Benjamin Scellier, Yoshua Bengio
Date: 2017-11-22 00:00:00

Recurrent Backpropagation and Equilibrium Propagation are supervised learning
algorithms for fixed point recurrent neural networks which differ in their
second phase. In the first phase, both algorithms converge to a fixed point
which corresponds to the configuration where the prediction is made. In the
second phase, Equilibrium Propagation relaxes to another nearby fixed point
corresponding to smaller prediction error, whereas Recurrent Backpropagation
uses a side network to compute error derivatives iteratively. In this work we
establish a close connection between these two algorithms. We show that, at
every moment in the second phase, the temporal derivatives of the neural
activities in Equilibrium Propagation are equal to the error derivatives
computed iteratively by Recurrent Backpropagation in the side network. This
work shows that it is not required to have a side network for the computation
of error derivatives, and supports the hypothesis that, in biological neural
networks, temporal derivatives of neural activities may code for error signals.
Multi-View Graph Convolutional Network and Its Applications on Neuroimage Analysis for Parkinson's Disease
Authors: Xi Sheryl Zhang, Lifang He, Kun Chen, Yuan Luo, Jiayu Zhou, Fei Wang
Date: 2018-05-22 00:00:00

Parkinson's Disease (PD) is one of the most prevalent neurodegenerative diseases that affects tens of millions of Americans. PD is highly progressive and heterogeneous. Quite a few studies have been conducted in recent years on predictive or disease progression modeling of PD using clinical and biomarkers data. Neuroimaging, as another important information source for neurodegenerative disease, has also arisen considerable interests from the PD community. In this paper, we propose a deep learning method based on Graph Convolutional Networks (GCN) for fusing multiple modalities of brain images in relationship prediction which is useful for distinguishing PD cases from controls. On Parkinson's Progression Markers Initiative (PPMI) cohort, our approach achieved $0.9537\pm 0.0587$ AUC, compared with $0.6443\pm 0.0223$ AUC achieved by traditional approaches such as PCA.
Super learning in the SAS system
Authors: Alexander P. Keil, Daniel Westreich, Jessie K Edwards, Stephen R Cole
Date: 2018-05-21 00:00:00
Tasks: BIG-bench Machine Learning, Causal Inference, Ensemble Learning

Background and objective: Stacking is an ensemble machine learning method that averages predictions from multiple other algorithms, such as generalized linear models and regression trees. An implementation of stacking, called super learning, has been developed as a general approach to supervised learning and has seen frequent usage, in part due to the availability of an R package. We develop super learning in the SAS software system using a new macro, and demonstrate its performance relative to the R package. Methods: Following previous work using the R SuperLearner package we assess the performance of super learning in a number of domains. We compare the R package with the new SAS macro in a small set of simulations assessing curve fitting in a predictive model as well in a set of 14 publicly available datasets to assess cross-validated accuracy. Results: Across the simulated data and the publicly available data, the SAS macro performed similarly to the R package, despite a different set of potential algorithms available natively in R and SAS. Conclusions: Our super learner macro performs as well as the R package at a number of tasks. Further, by extending the macro to include the use of R packages, the macro can leverage both the robust, enterprise oriented procedures in SAS and the nimble, cutting edge packages in R. In the spirit of ensemble learning, this macro extends the potential library of algorithms beyond a single software system and provides a simple avenue into machine learning in SAS.
A scene perception system for visually impaired based on object detection and classification using multi-modal DCNN
Authors: Baljit Kaur, Jhilik Bhattacharya
Date: 2018-05-22 00:00:00
Tasks: General Classification, object-detection, Object Detection

This paper represents a cost-effective scene perception system aimed towards
visually impaired individual. We use an odroid system integrated with an USB
camera and USB laser that can be attached on the chest. The system classifies
the detected objects along with its distance from the user and provides a voice
output. Experimental results provided in this paper use outdoor traffic scenes.
The object detection and classification framework exploits a multi-modal fusion
based faster RCNN using motion, sharpening and blurring filters for efficient
feature representation.
Mean Field Theory of Activation Functions in Deep Neural Networks
Authors: Mirco Milletarí, Thiparat Chotibut, Paolo E. Trevisanutto
Date: 2018-05-22 00:00:00
Tasks: General Classification

We present a Statistical Mechanics (SM) model of deep neural networks, connecting the energy-based and the feed forward networks (FFN) approach. We infer that FFN can be understood as performing three basic steps: encoding, representation validation and propagation. From the meanfield solution of the model, we obtain a set of natural activations -- such as Sigmoid, $\tanh$ and ReLu -- together with the state-of-the-art, Swish; this represents the expected information propagating through the network and tends to ReLu in the limit of zero noise.We study the spectrum of the Hessian on an associated classification task, showing that Swish allows for more consistent performances over a wider range of network architectures.
A Convolutional Feature Map based Deep Network targeted towards Traffic Detection and Classification
Authors: Baljit Kaur, Jhilik Bhattacharya
Date: 2018-05-22 00:00:00
Tasks: General Classification, Object, object-detection, Object Detection, Optical Flow Estimation

This research mainly emphasizes on traffic detection thus essentially
involving object detection and classification. The particular work discussed
here is motivated from unsatisfactory attempts of re-using well known
pre-trained object detection networks for domain specific data. In this course,
some trivial issues leading to prominent performance drop are identified and
ways to resolve them are discussed. For example, some simple yet relevant
tricks regarding data collection and sampling prove to be very beneficial.
Also, introducing a blur net to deal with blurred real time data is another
important factor promoting performance elevation. We further study the neural
network design issues for beneficial object classification and involve shared,
region-independent convolutional features. Adaptive learning rates to deal with
saddle points are also investigated and an average covariance matrix based
pre-conditioned approach is proposed. We also introduce the use of optical flow
features to accommodate orientation information. Experimental results
demonstrate that this results in a steady rise in the performance rate.
Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication
Authors: Felix Sattler, Simon Wiedemann, Klaus-Robert Müller, Wojciech Samek
Date: 2018-05-22 00:00:00
Tasks: Binarization, Deep Learning

Currently, progressively larger deep neural networks are trained on ever
growing data corpora. As this trend is only going to increase in the future,
distributed training schemes are becoming increasingly relevant. A major issue
in distributed training is the limited communication bandwidth between
contributing nodes or prohibitive communication cost in general. These
challenges become even more pressing, as the number of computation nodes
increases. To counteract this development we propose sparse binary compression
(SBC), a compression framework that allows for a drastic reduction of
communication cost for distributed training. SBC combines existing techniques
of communication delay and gradient sparsification with a novel binarization
method and optimal weight update encoding to push compression gains to new
limits. By doing so, our method also allows us to smoothly trade-off gradient
sparsity and temporal sparsity to adapt to the requirements of the learning
task. Our experiments show, that SBC can reduce the upstream communication on a
variety of convolutional and recurrent neural network architectures by more
than four orders of magnitude without significantly harming the convergence
speed in terms of forward-backward passes. For instance, we can train ResNet50
on ImageNet in the same number of iterations to the baseline accuracy, using
$\times 3531$ less bits or train it to a $1\%$ lower accuracy using $\times
37208$ less bits. In the latter case, the total upstream communication required
is cut from 125 terabytes to 3.35 gigabytes for every participating client.
Clinical Parameters Prediction for Gait Disorder Recognition
Authors: Soheil Esmaeilzadeh, Ouassim Khebzegga, Mehrad Moradshahi
Date: 2018-05-22 00:00:00
Tasks: Prediction

Being able to predict clinical parameters in order to diagnose gait disorders
in a patient is of great value in planning treatments. It is known that
\textit{decision parameters} such as cadence, step length, and walking speed
are critical in the diagnosis of gait disorders in patients. This project aims
to predict the decision parameters using two ways and afterwards giving advice
on whether a patient needs treatment or not. In one way, we use clinically
measured parameters such as Ankle Dorsiflexion, age, walking speed, step
length, stride length, weight over height squared (BMI) and etc. to predict the
decision parameters. In a second way, we use videos recorded from patient's
walking tests in a clinic in order to extract the coordinates of the joints of
the patient over time and predict the decision parameters. Finally, having the
decision parameters we pre-classify gait disorder intensity of a patient and as
the result make decisions on whether a patient needs treatment or not.
ADef: an Iterative Algorithm to Construct Adversarial Deformations
Authors: Rima Alaifari, Giovanni S. Alberti, Tandri Gauksson
Date: 2018-04-20 00:00:00
Proceeding: ICLR 2019 5
Tasks: Adversarial Attack, General Classification

While deep neural networks have proven to be a powerful tool for many
recognition and classification tasks, their stability properties are still not
well understood. In the past, image classifiers have been shown to be
vulnerable to so-called adversarial attacks, which are created by additively
perturbing the correctly classified image. In this paper, we propose the ADef
algorithm to construct a different kind of adversarial attack created by
iteratively applying small deformations to the image, found through a gradient
descent step. We demonstrate our results on MNIST with convolutional neural
networks and on ImageNet with Inception-v3 and ResNet-101.
On semi-supervised learning
Authors: Alejandro Cholaquidis, Ricardo Fraimand, Mariela Sued
Date: 2018-05-22 00:00:00
Tasks: General Classification

Semi-supervised learning deals with the problem of how, if possible, to take advantage of a huge amount of unclassified data, to perform a classification in situations when, typically, there is little labeled data. Even though this is not always possible (it depends on how useful, for inferring the labels, it would be to know the distribution of the unlabeled data), several algorithm have been proposed recently. %but in general they are not proved to outperform A new algorithm is proposed, that under almost necessary conditions, %and it is proved that it attains asymptotically the performance of the best theoretical rule as the amount of unlabeled data tends to infinity. The set of necessary assumptions, although reasonable, show that semi-supervised classification only works for very well conditioned problems. The focus is on understanding when and why semi-supervised learning works when the size of the initial training sample remains fixed and the asymptotic is on the size of the unlabeled data. The performance of the algorithm is assessed in the well known "Isolet" real-data of phonemes, where a strong dependence on the choice of the initial training sample is shown.
FAKEDETECTOR: Effective Fake News Detection with Deep Diffusive Neural Network
Authors: Jiawei Zhang, Bowen Dong, Philip S. Yu
Date: 2018-05-22 00:00:00
Tasks: Articles, Fake News Detection

In recent years, due to the booming development of online social networks, fake news for various commercial and political purposes has been appearing in large numbers and widespread in the online world. With deceptive words, online social network users can get infected by these online fake news easily, which has brought about tremendous effects on the offline society already. An important goal in improving the trustworthiness of information in online social networks is to identify the fake news timely. This paper aims at investigating the principles, methodologies and algorithms for detecting fake news articles, creators and subjects from online social networks and evaluating the corresponding performance. This paper addresses the challenges introduced by the unknown characteristics of fake news and diverse connections among news articles, creators and subjects. This paper introduces a novel automatic fake news credibility inference model, namely FAKEDETECTOR. Based on a set of explicit and latent features extracted from the textual information, FAKEDETECTOR builds a deep diffusive network model to learn the representations of news articles, creators and subjects simultaneously. Extensive experiments have been done on a real-world fake news dataset to compare FAKEDETECTOR with several state-of-the-art models, and the experimental results have demonstrated the effectiveness of the proposed model.
A Tropical Approach to Neural Networks with Piecewise Linear Activations
Authors: Vasileios Charisopoulos, Petros Maragos
Date: 2018-05-22 00:00:00

We present a new, unifying approach following some recent developments on the
complexity of neural networks with piecewise linear activations. We treat
neural network layers with piecewise linear activations as tropical
polynomials, which generalize polynomials in the so-called $(\max, +)$ or
tropical algebra, with possibly real-valued exponents. Motivated by the
discussion in (arXiv:1402.1869), this approach enables us to refine their upper
bounds on linear regions of layers with ReLU or leaky ReLU activations to
$\min\left\{ 2^m, \sum_{j=0}^n \binom{m}{j} \right\}$, where $n, m$ are the
number of inputs and outputs, respectively. Additionally, we recover their
upper bounds on maxout layers. Our work follows a novel path, exclusively under
the lens of tropical geometry, which is independent of the improvements
reported in (arXiv:1611.01491, arXiv:1711.02114). Finally, we present a
geometric approach for effective counting of linear regions using random
sampling in order to avoid the computational overhead of exact counting
approaches
EgoCoder: Intelligent Program Synthesis with Hierarchical Sequential Neural Network Model
Authors: Jiawei Zhang, Limeng Cui, Fisher B. Gouza
Date: 2018-05-22 00:00:00
Tasks: Program Synthesis, Text Generation

Programming has been an important skill for researchers and practitioners in
computer science and other related areas. To learn basic programing skills, a
long-time systematic training is usually required for beginners. According to a
recent market report, the computer software market is expected to continue
expanding at an accelerating speed, but the market supply of qualified software
developers can hardly meet such a huge demand. In recent years, the surge of
text generation research works provides the opportunities to address such a
dilemma through automatic program synthesis. In this paper, we propose to make
our try to solve the program synthesis problem from a data mining perspective.
To address the problem, a novel generative model, namely EgoCoder, will be
introduced in this paper. EgoCoder effectively parses program code into
abstract syntax trees (ASTs), where the tree nodes will contain the program
code/comment content and the tree structure can capture the program logic
flows. Based on a new unit model called Hsu, EgoCoder can effectively capture
both the hierarchical and sequential patterns in the program ASTs. Extensive
experiments will be done to compare EgoCoder with the state-of-the-art text
generation methods, and the experimental results have demonstrated the
effectiveness of EgoCoder in addressing the program synthesis problem.
CascadeCNN: Pushing the performance limits of quantisation
Authors: Alexandros Kouris, Stylianos I. Venieris, Christos-Savvas Bouganis
Date: 2018-05-22 00:00:00

This work presents CascadeCNN, an automated toolflow that pushes the
quantisation limits of any given CNN model, to perform high-throughput
inference by exploiting the computation time-accuracy trade-off. Without the
need for retraining, a two-stage architecture tailored for any given FPGA
device is generated, consisting of a low- and a high-precision unit. A
confidence evaluation unit is employed between them to identify misclassified
cases at run time and forward them to the high-precision unit or terminate
computation. Experiments demonstrate that CascadeCNN achieves a performance
boost of up to 55% for VGG-16 and 48% for AlexNet over the baseline design for
the same resource budget and accuracy.
UnibucKernel: A kernel-based learning method for complex word identification
Authors: Andrei M. Butnaru, Radu Tudor Ionescu
Date: 2018-03-20 00:00:00
Proceeding: WS 2018 6
Tasks: Binary Classification, Complex Word Identification, regression, Word Embeddings

In this paper, we present a kernel-based learning approach for the 2018
Complex Word Identification (CWI) Shared Task. Our approach is based on
combining multiple low-level features, such as character n-grams, with
high-level semantic features that are either automatically learned using word
embeddings or extracted from a lexical knowledge base, namely WordNet. After
feature extraction, we employ a kernel method for the learning phase. The
feature matrix is first transformed into a normalized kernel matrix. For the
binary classification task (simple versus complex), we employ Support Vector
Machines. For the regression task, in which we have to predict the complexity
level of a word (a word is more complex if it is labeled as complex by more
annotators), we employ v-Support Vector Regression. We applied our approach
only on the three English data sets containing documents from Wikipedia,
WikiNews and News domains. Our best result during the competition was the third
place on the English Wikipedia data set. However, in this paper, we also report
better post-competition results.
Adversarially Robust Training through Structured Gradient Regularization
Authors: Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, Thomas Hofmann
Date: 2018-05-22 00:00:00

We propose a novel data-dependent structured gradient regularizer to increase
the robustness of neural networks vis-a-vis adversarial perturbations. Our
regularizer can be derived as a controlled approximation from first principles,
leveraging the fundamental link between training with noise and regularization.
It adds very little computational overhead during learning and is simple to
implement generically in standard deep learning frameworks. Our experiments
provide strong evidence that structured gradient regularization can act as an
effective first line of defense against attacks based on low-level signal
corruption.
Efficient Stochastic Gradient Descent for Learning with Distributionally Robust Optimization
Authors: Soumyadip Ghosh, Mark Squillante, Ebisa Wollega
Date: 2018-05-22 00:00:00

Distributionally robust optimization (DRO) problems are increasingly seen as a viable method to train machine learning models for improved model generalization. These min-max formulations, however, are more difficult to solve. We therefore provide a new stochastic gradient descent algorithm to efficiently solve this DRO formulation. Our approach applies gradient descent to the outer minimization formulation and estimates the gradient of the inner maximization based on a sample average approximation. The latter uses a subset of the data in each iteration, progressively increasing the subset size to ensure convergence. Theoretical results include establishing the optimal manner for growing the support size to balance a fundamental tradeoff between stochastic error and computational effort. Empirical results demonstrate the significant benefits of our approach over previous work, and also illustrate how learning with DRO can improve generalization.
Adversarial Training of Word2Vec for Basket Completion
Authors: Ugo Tanielian, Mike Gartrell, Flavian vasile
Date: 2018-05-22 00:00:00
Tasks: Language Modeling, Language Modelling, Word Similarity

In recent years, the Word2Vec model trained with the Negative Sampling loss
function has shown state-of-the-art results in a number of machine learning
tasks, including language modeling tasks, such as word analogy and word
similarity, and in recommendation tasks, through Prod2Vec, an extension that
applies to modeling user shopping activity and user preferences. Several
methods that aim to improve upon the standard Negative Sampling loss have been
proposed. In our paper we pursue more sophisticated Negative Sampling, by
leveraging ideas from the field of Generative Adversarial Networks (GANs), and
propose Adversarial Negative Sampling. We build upon the recent progress made
in stabilizing the training objective of GANs in the discrete data setting, and
introduce a new GAN-Word2Vec model.We evaluate our model on the task of basket
completion, and show significant improvements in performance over Word2Vec
trained using standard loss functions, including Noise Contrastive Estimation
and Negative Sampling.
Parsimonious Bayesian deep networks
Authors: Mingyuan Zhou
Date: 2018-05-22 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Model Selection

Combining Bayesian nonparametrics and a forward model selection strategy, we
construct parsimonious Bayesian deep networks (PBDNs) that infer
capacity-regularized network architectures from the data and require neither
cross-validation nor fine-tuning when training the model. One of the two
essential components of a PBDN is the development of a special infinite-wide
single-hidden-layer neural network, whose number of active hidden units can be
inferred from the data. The other one is the construction of a greedy
layer-wise learning algorithm that uses a forward model selection criterion to
determine when to stop adding another hidden layer. We develop both Gibbs
sampling and stochastic gradient descent based maximum a posteriori inference
for PBDNs, providing state-of-the-art classification accuracy and interpretable
data subtypes near the decision boundaries, while maintaining low computational
complexity for out-of-sample prediction.