pwc-archive/papers-with-abstracts huggingface dataset:
$S^{2}$-LBI: Stochastic Split Linearized Bregman Iterations for Parsimonious Deep Learning
Authors: Yanwei Fu, Donghao Li, Xinwei Sun, Shun Zhang, Yizhou Wang, Yuan YAO
Date: 2019-04-24 00:00:00
Tasks: Computational Efficiency, Model Selection

This paper proposes a novel Stochastic Split Linearized Bregman Iteration
($S^{2}$-LBI) algorithm to efficiently train the deep network. The $S^{2}$-LBI
introduces an iterative regularization path with structural sparsity. Our
$S^{2}$-LBI combines the computational efficiency of the LBI, and model
selection consistency in learning the structural sparsity. The computed
solution path intrinsically enables us to enlarge or simplify a network, which
theoretically, is benefited from the dynamics property of our $S^{2}$-LBI
algorithm. The experimental results validate our $S^{2}$-LBI on MNIST and
CIFAR-10 dataset. For example, in MNIST, we can either boost a network with
only 1.5K parameters (1 convolutional layer of 5 filters, and 1 FC layer),
achieves 98.40\% recognition accuracy; or we simplify $82.5\%$ of parameters in
LeNet-5 network, and still achieves the 98.47\% recognition accuracy. In
addition, we also have the learning results on ImageNet, which will be added in
the next version of our report.
Variational approach to unsupervised learning
Authors: Swapnil Nitin Shah
Date: 2019-04-24 00:00:00

Deep belief networks are used extensively for unsupervised stochastic
learning on large datasets. Compared to other deep learning approaches their
layer-by-layer learning makes them highly scalable. Unfortunately, the
principles by which they achieve efficient learning are not well understood.
Numerous attempts have been made to explain their efficiency and applicability
to a wide class of learning problems in terms of principles drawn from
cognitive psychology, statistics, information theory, and more recently
physics, but quite often these imported principles lack strong scientific
foundation. Here we demonstrate how one can arrive at convolutional deep belief
networks as potential solution to unsupervised learning problems without making
assumptions about the underlying framework. To do this, we exploit the notion
of symmetry that is fundamental in machine learning, physics and other fields,
utilizing the particular form of the functional renormalization group in
physics.
Unsupervised Assignment Flow: Label Learning on Feature Manifolds by Spatially Regularized Geometric Assignment
Authors: Artjom Zern, Matthias Zisler, Stefania Petra, Christoph Schnörr
Date: 2019-04-24 00:00:00
Tasks: Clustering

This paper introduces the unsupervised assignment flow that couples the assignment flow for supervised image labeling with Riemannian gradient flows for label evolution on feature manifolds. The latter component of the approach encompasses extensions of state-of-the-art clustering approaches to manifold-valued data. Coupling label evolution with the spatially regularized assignment flow induces a sparsifying effect that enables to learn compact label dictionaries in an unsupervised manner. Our approach alleviates the requirement for supervised labeling to have proper labels at hand, because an initial set of labels can evolve and adapt to better values while being assigned to given data. The separation between feature and assignment manifolds enables the flexible application which is demonstrated for three scenarios with manifold-valued features. Experiments demonstrate a beneficial effect in both directions: adaptivity of labels improves image labeling, and steering label evolution by spatially regularized assignments leads to proper labels, because the assignment flow for supervised labeling is exactly used without any approximation for label learning.
Optical machine learning with incoherent light and a single-pixel detector
Authors: Shuming Jiao, Jun Feng, Yang Gao, Ting Lei, Zhenwei Xie, Xiaocong Yuan
Date: 2019-04-24 00:00:00
Tasks: BIG-bench Machine Learning

An optical diffractive neural network (DNN) can be implemented with a cascaded phase mask architecture. Like an optical computer, the system can perform machine learning tasks such as number digit recognition in an all-optical manner. However, the system can only work under coherent light illumination and the precision requirement in practical experiments is quite high. This paper proposes an optical machine learning framework based on single-pixel imaging (MLSPI). The MLSPI system can perform the same linear pattern recognition task as DNN. Furthermore, it can work under incoherent lighting conditions, has lower experimental complexity and can be easily programmable.
Wearable-based Parkinson's Disease Severity Monitoring using Deep Learning
Authors: Jann Goschenhofer, Franz MJ Pfister, Kamer Ali Yuksel, Bernd Bischl, Urban Fietzek, Janek Thomas
Date: 2019-04-24 00:00:00
Tasks: Deep Learning, General Classification, regression, Time Series, Time Series Analysis, Time Series Classification, Transfer Learning

One major challenge in the medication of Parkinson's disease is that the
severity of the disease, reflected in the patients' motor state, cannot be
measured using accessible biomarkers. Therefore, we develop and examine a
variety of statistical models to detect the motor state of such patients based
on sensor data from a wearable device. We find that deep learning models
consistently outperform a classical machine learning model applied on
hand-crafted features in this time series classification task. Furthermore, our
results suggest that treating this problem as a regression instead of an
ordinal regression or a classification task is most appropriate. For consistent
model evaluation and training, we adopt the leave-one-subject-out validation
scheme to the training of deep learning models. We also employ a
class-weighting scheme to successfully mitigate the problem of high multi-class
imbalances in this domain. In addition, we propose a customized performance
measure that reflects the requirements of the involved medical staff on the
model. To solve the problem of limited availability of high quality training
data, we propose a transfer learning technique which helps to improve model
performance substantially. Our results suggest that deep learning techniques
offer a high potential to autonomously detect motor states of patients with
Parkinson's disease.
Learning Bodily and Temporal Attention in Protective Movement Behavior Detection
Authors: Chongyang Wang, Min Peng, Temitayo A. Olugbade, Nicholas D. Lane, Amanda C. De C. Williams, Nadia Bianchi-Berthouze
Date: 2019-04-24 00:00:00

For people with chronic pain, the assessment of protective behavior during physical functioning is essential to understand their subjective pain-related experiences (e.g., fear and anxiety toward pain and injury) and how they deal with such experiences (avoidance or reliance on specific body joints), with the ultimate goal of guiding intervention. Advances in deep learning (DL) can enable the development of such intervention. Using the EmoPain MoCap dataset, we investigate how attention-based DL architectures can be used to improve the detection of protective behavior by capturing the most informative temporal and body configurational cues characterizing specific movements and the strategies used to perform them. We propose an end-to-end deep learning architecture named BodyAttentionNet (BANet). BANet is designed to learn temporal and bodily parts that are more informative to the detection of protective behavior. The approach addresses the variety of ways people execute a movement (including healthy people) independently of the type of movement analyzed. Through extensive comparison experiments with other state-of-the-art machine learning techniques used with motion capture data, we show statistically significant improvements achieved by using these attention mechanisms. In addition, the BANet architecture requires a much lower number of parameters than the state of the art for comparable if not higher performances.
Semantic Drift in Multilingual Representations
Authors: Lisa Beinborn, Rochelle Choenni
Date: 2019-04-24 00:00:00
Proceeding: CL (ACL) 2020 9
Tasks: Sentence

Multilingual representations have mostly been evaluated based on their performance on specific tasks. In this article, we look beyond engineering goals and analyze the relations between languages in computational representations. We introduce a methodology for comparing languages based on their organization of semantic concepts. We propose to conduct an adapted version of representational similarity analysis of a selected set of concepts in computational multilingual representations. Using this analysis method, we can reconstruct a phylogenetic tree that closely resembles those assumed by linguistic experts. These results indicate that multilingual distributional representations which are only trained on monolingual text and bilingual dictionaries preserve relations between languages without the need for any etymological information. In addition, we propose a measure to identify semantic drift between language families. We perform experiments on word-based and sentence-based multilingual models and provide both quantitative results and qualitative examples. Analyses of semantic drift in multilingual representations can serve two purposes: they can indicate unwanted characteristics of the computational models and they provide a quantitative means to study linguistic phenomena across languages. The code is available at https://github.com/beinborn/SemanticDrift.
Native Banach spaces for splines and variational inverse problems
Authors: Michael Unser, Julien Fageot
Date: 2019-04-24 00:00:00

We propose a systematic construction of native Banach spaces for general
spline-admissible operators ${\rm L}$. In short, the native space for ${\rm L}$
and the (dual) norm $\|\cdot\|_{\mathcal{X}'}$ is the largest space of
functions $f: \mathbb{R}^d \to \mathbb{R}$ such that $\|{\rm L}
f\|_{\mathcal{X}'}<\infty$, subject to the constraint that the
growth-restricted null space of ${\rm L}$be finite-dimensional. This space,
denoted by $\mathcal{X}'_{\rm L}$, is specified as the dual of the pre-native
space $\mathcal{X}_{\rm L}$, which is itself obtained through a suitable
completion process. The main difference with prior constructions (e.g.,
reproducing kernel Hilbert spaces) is that our approach involves test functions
rather than sums of atoms (e.g, kernels), which makes it applicable to a much
broader class of norms, including total variation. Under specific admissibility
and compatibility hypotheses, we lay out the direct-sum topology of
$\mathcal{X}_{\rm L}$ and $\mathcal{X}'_{\rm L}$, and identify the whole family
of equivalent norms. Our construction ensures that the native space and its
pre-dual are endowed with a fundamental Schwartz-Banach property. In practical
terms, this means that $\mathcal{X}'_{\rm L}$ is rich enough to reproduce any
function with an arbitrary degree of precision.
Automatic cephalometric landmarks detection on frontal faces: an approach based on supervised learning techniques
Authors: Lucas Faria Porto, Laise Nascimento Correia Lima, Marta Flores, Andrea Valsecchi, Oscar Ibanez, Carlos Eduardo Machado Palhares, Flavio de Barros Vidal
Date: 2019-04-24 00:00:00
Tasks: Descriptive, Facial Landmark Detection

Facial landmarks are employed in many research areas such as facial
recognition, craniofacial identification, age and sex estimation among the most
important. In the forensic field, the focus is on the analysis of a particular
set of facial landmarks, defined as cephalometric landmarks. Previous works
demonstrated that the descriptive adequacy of these anatomical references for
an indirect application (photo-anthropometric description) increased the
marking precision of these points, contributing to a greater reliability of
these analyzes. However, most of them are performed manually and all of them
are subjectivity inherent to the expert examiners. In this sense, the purpose
of this work is the development and validation of automatic techniques to
detect cephalometric landmarks from digital images of frontal faces in forensic
field. The presented approach uses a combination of computer vision and image
processing techniques within a supervised learning procedures. The proposed
methodology obtains similar precision to a group of human manual cephalometric
reference markers and result to be more accurate against others
state-of-the-art facial landmark detection frameworks. It achieves a normalized
mean distance (in pixel) error of 0.014, similar to the mean inter-expert
dispersion (0.009) and clearly better than other automatic approaches also
analyzed along of this work (0.026 and 0.101).
Mercator: uncovering faithful hyperbolic embeddings of complex networks
Authors: Guillermo García-Pérez, Antoine Allard, M. Ángeles Serrano, Marián Boguñá
Date: 2019-04-24 00:00:00
Tasks: BIG-bench Machine Learning

We introduce Mercator, a reliable embedding method to map real complex
networks into their hyperbolic latent geometry. The method assumes that the
structure of networks is well described by the Popularity$\times$Similarity
$\mathbb{S}^1/\mathbb{H}^2$ static geometric network model, which can
accommodate arbitrary degree distributions and reproduces many pivotal
properties of real networks, including self-similarity patterns. The algorithm
mixes machine learning and maximum likelihood approaches to infer the
coordinates of the nodes in the underlying hyperbolic disk with the best
matching between the observed network topology and the geometric model. In its
fast mode, Mercator uses a model-adjusted machine learning technique performing
dimensional reduction to produce a fast and accurate map, whose quality already
outperform other embedding algorithms in the literature. In the refined
Mercator mode, the fast-mode embedding result is taken as an initial condition
in a Maximum Likelihood estimation, which significantly improves the quality of
the final embedding. Apart from its accuracy as an embedding tool, Mercator has
the clear advantage of systematically inferring not only node orderings, or
angular positions, but also the hidden degrees and global model parameters, and
has the ability to embed networks with arbitrary degree distributions. Overall,
our results suggest that mixing machine learning and maximum likelihood
techniques in a model-dependent framework can boost the meaningful mapping of
complex networks.
Three Methods for Training on Bandit Feedback
Authors: Dmytro Mykhaylov, David Rohde, Flavian vasile, Martin Bompaire, Olivier Jeunen
Date: 2019-04-24 00:00:00
Tasks: Recommendation Systems

There are three quite distinct ways to train a machine learning model on recommender system logs. The first method is to model the reward prediction for each possible recommendation to the user, at the scoring time the best recommendation is found by computing an argmax over the personalized recommendations. This method obeys principles such as the conditionality principle and the likelihood principle. A second method is useful when the model does not fit reality and underfits. In this case, we can use the fact that we know the distribution of historical recommendations (concentrated on previously identified good actions with some exploration) to adjust the errors in the fit to be evenly distributed over all actions. Finally, the inverse propensity score can be used to produce an estimate of the decision rules expected performance. The latter two methods violate the conditionality and likelihood principle but are shown to have good performance in certain settings. In this paper we review the literature around this fundamental, yet often overlooked choice and do some experiments using the RecoGym simulation environment.
Machine learning for long-distance quantum communication
Authors: Julius Wallnöfer, Alexey A. Melnikov, Wolfgang Dür, Hans J. Briegel
Date: 2019-04-24 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Reinforcement Learning

Machine learning can help us in solving problems in the context big data analysis and classification, as well as in playing complex games such as Go. But can it also be used to find novel protocols and algorithms for applications such as large-scale quantum communication? Here we show that machine learning can be used to identify central quantum protocols, including teleportation, entanglement purification and the quantum repeater. These schemes are of importance in long-distance quantum communication, and their discovery has shaped the field of quantum information processing. However, the usefulness of learning agents goes beyond the mere re-production of known protocols; the same approach allows one to find improved solutions to long-distance communication problems, in particular when dealing with asymmetric situations where channel noise and segment distance are non-uniform. Our findings are based on the use of projective simulation, a model of a learning agent that combines reinforcement learning and decision making in a physically motivated framework. The learning agent is provided with a universal gate set, and the desired task is specified via a reward scheme. From a technical perspective, the learning agent has to deal with stochastic environments and reactions. We utilize an idea reminiscent of hierarchical skill acquisition, where solutions to sub-problems are learned and re-used in the overall scheme. This is of particular importance in the development of long-distance communication schemes, and opens the way for using machine learning in the design and implementation of quantum networks.
Simultaneous regression and feature learning for facial landmarking
Authors: Janez Križaj, Peter Peer, Vitomir Štruc, Simon Dobrišek
Date: 2019-04-24 00:00:00
Tasks: Attribute, Face Alignment, regression

Face alignment (or facial landmarking) is an important task in many
face-related applications, ranging from registration, tracking and animation to
higher-level classification problems such as face, expression or attribute
recognition. While several solutions have been presented in the literature for
this task so far, reliably locating salient facial features across a wide range
of posses still remains challenging. To address this issue, we propose in this
paper a novel method for automatic facial landmark localization in 3D face data
designed specifically to address appearance variability caused by significant
pose variations. Our method builds on recent cascaded-regression-based methods
to facial landmarking and uses a gating mechanism to incorporate multiple
linear cascaded regression models each trained for a limited range of poses
into a single powerful landmarking model capable of processing arbitrary posed
input data. We develop two distinct approaches around the proposed gating
mechanism: i) the first uses a gated multiple ridge descent (GRID) mechanism in
conjunction with established (hand-crafted) HOG features for face alignment and
achieves state-of-the-art landmarking performance across a wide range of facial
poses, ii) the second simultaneously learns multiple-descent directions as well
as binary features (SMUF) that are optimal for the alignment tasks and in
addition to competitive landmarking results also ensures extremely rapid
processing. We evaluate both approaches in rigorous experiments on several
popular datasets of 3D face images, i.e., the FRGCv2 and Bosphorus 3D Face
datasets and image collections F and G from the University of Notre Dame. The
results of our evaluation show that both approaches are competitive in
comparison to the state-of-the-art, while exhibiting considerable robustness to
pose variations.
Latent Variable Session-Based Recommendation
Authors: David Rohde, Stephen Bonner
Date: 2019-04-24 00:00:00
Proceeding: pproximateinference AABI Symposium 2019 12
Tasks: Feature Engineering, Session-Based Recommendations

Session based recommendation provides an attractive alternative to the traditional feature engineering approach to recommendation. Feature engineering approaches require hand tuned features of the users history to be created to produce a context vector. In contrast a session based approach is able to dynamically model the users state as they act. We present a probabilistic framework for session based recommendation. A latent variable for the user state is updated as the user views more items and we learn more about their interests. The latent variable model is conceptually simple and elegant; yet requires sophisticated computational technique to approximate the integral over the latent variable. We provide computational solutions using both the re-parameterization trick and also using the Bouchard bound for the softmax function, we further explore employing a variational auto-encoder and a variational Expectation-Maximization algorithm for tightening the variational bound. The model performs well against a number of baselines. The intuitive nature of the model allows an elegant formulation combining correlations between items and their popularity and that sheds light on other popular recommendation methods. An attractive feature of the latent variable approach is that, as the user continues to act, the posterior on the user's state tightens reflecting the recommender system's increased knowledge about that user.
Informative sample generation using class aware generative adversarial networks for classification of chest Xrays
Authors: Behzad Bozorgtabar, Dwarikanath Mahapatra, Hendrik von Teng, Alexander Pollinger, Lukas Ebner, Jean-Phillipe Thiran, Mauricio Reyes
Date: 2019-04-24 00:00:00
Tasks: Active Learning, Data Augmentation, General Classification, Generative Adversarial Network

Training robust deep learning (DL) systems for disease detection from medical
images is challenging due to limited images covering different disease types
and severity. The problem is especially acute, where there is a severe class
imbalance. We propose an active learning (AL) framework to select most
informative samples for training our model using a Bayesian neural network.
Informative samples are then used within a novel class aware generative
adversarial network (CAGAN) to generate realistic chest xray images for data
augmentation by transferring characteristics from one class label to another.
Experiments show our proposed AL framework is able to achieve state-of-the-art
performance by using about $35\%$ of the full dataset, thus saving significant
time and effort over conventional methods.
Some Limit Properties of Markov Chains Induced by Stochastic Recursive Algorithms
Authors: Abhishek Gupta, Hao Chen, Jianzong Pi, Gaurav Tendolkar
Date: 2019-04-24 00:00:00

Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic programming algorithms for solving Markov decision problems. These recursive stochastic algorithms approximate certain contraction operators and can be viewed within the framework of iterated random operators. Accordingly, we consider iterated random operators over a Polish space that simulate iterated contraction operator over that Polish space. Assume that the iterated random operators are indexed by certain batch sizes such that as batch sizes grow to infinity, each realization of the random operator converges (in some sense) to the contraction operator it is simulating. We show that starting from the same initial condition, the distribution of the random sequence generated by the iterated random operators converges weakly to the trajectory generated by the contraction operator. We further show that under certain conditions, the time average of the random sequence converges to the spatial mean of the invariant distribution. We then apply these results to logistic regression, empirical value iteration, and empirical Q value iteration for finite state finite action MDPs to illustrate the general theory develop here.