pwc-archive/papers-with-abstracts huggingface dataset:
Sparse Deep Predictive Coding captures contour integration capabilities of the early visual system
Authors: Victor Boutin, Angelo Franciosini, Frederic Chavane, Franck Ruffier, Laurent Perrinet
Date: 2019-02-20 00:00:00
Tasks: Decision Making

Both neurophysiological and psychophysical experiments have pointed out the crucial role of recurrent and feedback connections to process context-dependent information in the early visual cortex. While numerous models have accounted for feedback effects at either neural or representational level, none of them were able to bind those two levels of analysis. Is it possible to describe feedback effects at both levels using the same model? We answer this question by combining Predictive Coding (PC) and Sparse Coding (SC) into a hierarchical and convolutional framework. In this Sparse Deep Predictive Coding (SDPC) model, the SC component models the internal recurrent processing within each layer, and the PC component describes the interactions between layers using feedforward and feedback connections. Here, we train a 2-layered SDPC on two different databases of images, and we interpret it as a model of the early visual system (V1 & V2). We first demonstrate that once the training has converged, SDPC exhibits oriented and localized receptive fields in V1 and more complex features in V2. Second, we analyze the effects of feedback on the neural organization beyond the classical receptive field of V1 neurons using interaction maps. These maps are similar to association fields and reflect the Gestalt principle of good continuation. We demonstrate that feedback signals reorganize interaction maps and modulate neural activity to promote contour integration. Third, we demonstrate at the representational level that the SDPC feedback connections are able to overcome noise in input images. Therefore, the SDPC captures the association field principle at the neural level which results in better disambiguation of blurred images at the representational level.
Machine Learning Algorithms for Classification of Microcirculation Images from Septic and Non-Septic Patients
Authors: Perikumar Javia, Aman Rana, Nathan Shapiro, Pratik Shah
Date: 2018-10-24 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Diagnostic, General Classification

Sepsis is a life-threatening disease and one of the major causes of death in
hospitals. Imaging of microcirculatory dysfunction is a promising approach for
automated diagnosis of sepsis. We report a machine learning classifier capable
of distinguishing non-septic and septic images from dark field microcirculation
videos of patients. The classifier achieves an accuracy of 89.45%. The area
under the receiver operating characteristics of the classifier was 0.92, the
precision was 0.92 and the recall was 0.84. Codes representing the learned
feature space of trained classifier were visualized using t-SNE embedding and
were separable and distinguished between images from critically ill and
non-septic patients. Using an unsupervised convolutional autoencoder,
independent of the clinical diagnosis, we also report clustering of learned
features from a compressed representation associated with healthy images and
those with microcirculatory dysfunction. The feature space used by our trained
classifier to distinguish between images from septic and non-septic patients
has potential diagnostic application.
Random Search and Reproducibility for Neural Architecture Search
Authors: Liam Li, Ameet Talwalkar
Date: 2019-02-20 00:00:00
Tasks: Hyperparameter Optimization, Neural Architecture Search

Neural architecture search (NAS) is a promising research direction that has the potential to replace expert-designed networks with learned, task-specific architectures. In this work, in order to help ground the empirical results in this field, we propose new NAS baselines that build off the following observations: (i) NAS is a specialized hyperparameter optimization problem; and (ii) random search is a competitive baseline for hyperparameter optimization. Leveraging these observations, we evaluate both random search with early-stopping and a novel random search with weight-sharing algorithm on two standard NAS benchmarks---PTB and CIFAR-10. Our results show that random search with early-stopping is a competitive NAS baseline, e.g., it performs at least as well as ENAS, a leading NAS method, on both benchmarks. Additionally, random search with weight-sharing outperforms random search with early-stopping, achieving a state-of-the-art NAS result on PTB and a highly competitive result on CIFAR-10. Finally, we explore the existing reproducibility issues of published NAS results. We note the lack of source material needed to exactly reproduce these results, and further discuss the robustness of published results given the various sources of variability in NAS experimental setups. Relatedly, we provide all information (code, random seeds, documentation) needed to exactly reproduce our results, and report our random search with weight-sharing results for each benchmark on multiple runs.
Filtering Point Targets via Online Learning of Motion Models
Authors: Mehryar Emambakhsh, Alessandro Bay, Eduard Vazquez
Date: 2019-02-20 00:00:00
Tasks: regression

Filtering point targets in highly cluttered and noisy data frames can be very
challenging, especially for complex target motions. Fixed motion models can
fail to provide accurate predictions, while learning based algorithm can be
difficult to design (due to the variable number of targets), slow to train and
dependent on separate train/test steps. To address these issues, this paper
proposes a multi-target filtering algorithm which learns the motion models, on
the fly, using a recurrent neural network with a long short-term memory
architecture, as a regression block. The target state predictions are then
corrected using a novel data association algorithm, with a low computational
complexity. The proposed algorithm is evaluated over synthetic and real point
target filtering scenarios, demonstrating a remarkable performance over highly
cluttered data sequences.
Adaptive Iterative Hessian Sketch via A-Optimal Subsampling
Authors: Aijun Zhang, Hengtao Zhang, Guosheng Yin
Date: 2019-02-20 00:00:00

Iterative Hessian sketch (IHS) is an effective sketching method for modeling large-scale data. It was originally proposed by Pilanci and Wainwright (2016; JMLR) based on randomized sketching matrices. However, it is computationally intensive due to the iterative sketch process. In this paper, we analyze the IHS algorithm under the unconstrained least squares problem setting, then propose a deterministic approach for improving IHS via A-optimal subsampling. Our contributions are three-fold: (1) a good initial estimator based on the A-optimal design is suggested; (2) a novel ridged preconditioner is developed for repeated sketching; and (3) an exact line search method is proposed for determining the optimal step length adaptively. Extensive experimental results demonstrate that our proposed A-optimal IHS algorithm outperforms the existing accelerated IHS methods.
advertorch v0.1: An Adversarial Robustness Toolbox based on PyTorch
Authors: Gavin Weiguang Ding, Luyu Wang, Xiaomeng Jin
Date: 2019-02-20 00:00:00
Tasks: Adversarial Attack, Adversarial Defense, Adversarial Robustness

advertorch is a toolbox for adversarial robustness research. It contains
various implementations for attacks, defenses and robust training methods.
advertorch is built on PyTorch (Paszke et al., 2017), and leverages the
advantages of the dynamic computational graph to provide concise and efficient
reference implementations. The code is licensed under the LGPL license and is
open sourced at https://github.com/BorealisAI/advertorch .
Phoneme Level Language Models for Sequence Based Low Resource ASR
Authors: Siddharth Dalmia, Xinjian Li, Alan W. black, Florian Metze
Date: 2019-02-20 00:00:00
Tasks: Language Modeling, Language Modelling

Building multilingual and crosslingual models help bring different languages
together in a language universal space. It allows models to share parameters
and transfer knowledge across languages, enabling faster and better adaptation
to a new language. These approaches are particularly useful for low resource
languages. In this paper, we propose a phoneme-level language model that can be
used multilingually and for crosslingual adaptation to a target language. We
show that our model performs almost as well as the monolingual models by using
six times fewer parameters, and is capable of better adaptation to languages
not seen during training in a low resource scenario. We show that these
phoneme-level language models can be used to decode sequence based
Connectionist Temporal Classification (CTC) acoustic model outputs to obtain
comparable word error rates with Weighted Finite State Transducer (WFST) based
decoding in Babel languages. We also show that these phoneme-level language
models outperform WFST decoding in various low-resource conditions like
adapting to a new language and domain mismatch between training and testing
data.
Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPs
Authors: Marek Petrik, Reazul Hasan Russell
Date: 2019-02-20 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Bayesian Inference, Position, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Robust MDPs (RMDPs) can be used to compute policies with provable worst-case
guarantees in reinforcement learning. The quality and robustness of an RMDP
solution are determined by the ambiguity set---the set of plausible transition
probabilities---which is usually constructed as a multi-dimensional confidence
region. Existing methods construct ambiguity sets as confidence regions using
concentration inequalities which leads to overly conservative solutions. This
paper proposes a new paradigm that can achieve better solutions with the same
robustness guarantees without using confidence regions as ambiguity sets. To
incorporate prior knowledge, our algorithms optimize the size and position of
ambiguity sets using Bayesian inference. Our theoretical analysis shows the
safety of the proposed method, and the empirical results demonstrate its
practical promise.
Point cloud denoising based on tensor Tucker decomposition
Authors: Jianze Li, Xiao-Ping Zhang, Tuan Tran
Date: 2019-02-20 00:00:00
Tasks: Denoising

In this paper, we propose a new algorithm for point cloud denoising based on the tensor Tucker decomposition. We first represent the local surface patches of a noisy point cloud to be matrices by their distances to a reference point, and stack the similar patch matrices to be a 3rd order tensor. Then we use the Tucker decomposition to compress this patch tensor to be a core tensor of smaller size. We consider this core tensor as the frequency domain and remove the noise by manipulating the hard thresholding. Finally, all the fibers of the denoised patch tensor are placed back, and the average is taken if there are more than one estimators overlapped. The experimental evaluation shows that the proposed algorithm outperforms the state-of-the-art graph Laplacian regularized (GLR) algorithm when the Gaussian noise is high ($\sigma=0.1$), and the GLR algorithm is better in lower noise cases ($\sigma=0.04, 0.05, 0.08$).
Fast, Small, and Simple Document Listing on Repetitive Text Collections
Authors: Dustin Cobas, Gonzalo Navarro
Date: 2019-02-20 00:00:00
Tasks: Retrieval

Document listing on string collections is the task of finding all documents
where a pattern appears. It is regarded as the most fundamental document
retrieval problem, and is useful in various applications. Many of the
fastest-growing string collections are composed of very similar documents, such
as versioned code and document collections, genome repositories, etc. Plain
pattern-matching indexes designed for repetitive text collections achieve
orders-of-magnitude reductions in space. Instead, there are not many analogous
indexes for document retrieval. In this paper we present a simple document
listing index for repetitive string collections of total length $n$ that lists
the $ndoc$ distinct documents where a pattern of length $m$ appears in time
$\mathcal{O}(m+ndoc \cdot \log n)$. We exploit the repetitiveness of the
document array (i.e., the suffix array coarsened to document identifiers) to
grammar-compress it while precomputing the answers to nonterminals, and store
them in grammar-compressed form as well. Our experimental results show that our
index sharply outperforms existing alternatives in the space/time tradeoff map.
Sparsity Constrained Distributed Unmixing of Hyperspectral Data
Authors: Sara Khoshsokhan, Roozbeh Rajabi, Hadi Zayyani
Date: 2019-02-20 00:00:00
Tasks: Distributed Optimization

Spectral unmixing (SU) is a technique to characterize mixed pixels in
hyperspectral images measured by remote sensors. Most of the spectral unmixing
algorithms are developed using the linear mixing models. To estimate endmembers
and fractional abundance matrices in a blind problem, nonnegative matrix
factorization (NMF) and its developments are widely used in the SU problem. One
of the constraints which was added to NMF is sparsity, that was regularized by
Lq norm. In this paper, a new algorithm based on distributed optimization is
suggested for spectral unmixing. In the proposed algorithm, a network including
single-node clusters is employed. Each pixel in the hyperspectral images is
considered as a node in this network. The sparsity constrained distributed
unmixing is optimized with diffusion least mean p-power (LMP) strategy, and
then the update equations for fractional abundance and signature matrices are
obtained. Afterwards the proposed algorithm is analyzed for different values of
LMP power and Lq norms. Simulation results based on defined performance metrics
illustrate the advantage of the proposed algorithm in spectral unmixing of
hyperspectral data compared with other methods.
TomoGAN: Low-Dose Synchrotron X-Ray Tomography with Generative Adversarial Networks
Authors: Zhengchun Liu, Tekin Bicer, Rajkumar Kettimuthu, Doga Gursoy, Francesco De Carlo, Ian Foster
Date: 2019-02-20 00:00:00
Tasks: Denoising

Synchrotron-based x-ray tomography is a noninvasive imaging technique that allows for reconstructing the internal structure of materials at high spatial resolutions from tens of micrometers to a few nanometers. In order to resolve sample features at smaller length scales, however, a higher radiation dose is required. Therefore, the limitation on the achievable resolution is set primarily by noise at these length scales. We present \TOMOGAN{}, a denoising technique based on generative adversarial networks, for improving the quality of reconstructed images for low-dose imaging conditions. We evaluate our approach in two photon-budget-limited experimental conditions: (1) sufficient number of low-dose projections (based on Nyquist sampling), and (2) insufficient or limited number of high-dose projections. In both cases the angular sampling is assumed to be isotropic, and the photon budget throughout the experiment is fixed based on the maximum allowable radiation dose on the sample. Evaluation with both simulated and experimental datasets shows that our approach can significantly reduce noise in reconstructed images, improving the structural similarity score of simulation and experimental data from 0.18 to 0.9 and from 0.18 to 0.41, respectively. Furthermore, the quality of the reconstructed images with filtered back projection followed by our denoising approach exceeds that of reconstructions with the simultaneous iterative reconstruction technique, showing the computational superiority of our approach.
Where Do Human Heuristics Come From?
Authors: Marcel Binz, Dominik Endres
Date: 2019-02-20 00:00:00
Tasks: Decision Making, Meta-Learning, Variational Inference

Human decision-making deviates from the optimal solution, that maximizes cumulative rewards, in many situations. Here we approach this discrepancy from the perspective of bounded rationality and our goal is to provide a justification for such seemingly sub-optimal strategies. More specifically we investigate the hypothesis, that humans do not know optimal decision-making algorithms in advance, but instead employ a learned, resource-bounded approximation. The idea is formalized through combining a recently proposed meta-learning model based on Recurrent Neural Networks with a resource-bounded objective. The resulting approach is closely connected to variational inference and the Minimum Description Length principle. Empirical evidence is obtained from a two-armed bandit task. Here we observe patterns in our family of models that resemble differences between individual human participants.
Emulating Human Developmental Stages with Bayesian Neural Networks
Authors: Marcel Binz, Dominik Endres
Date: 2019-02-20 00:00:00

We compare the acquisition of knowledge in humans and machines. Research from
the field of developmental psychology indicates, that human-employed hypothesis
are initially guided by simple rules, before evolving into more complex
theories. This observation is shared across many tasks and domains. We
investigate whether stages of development in artificial learning systems are
based on the same characteristics. We operationalize developmental stages as
the size of the data-set, on which the artificial system is trained. For our
analysis we look at the developmental progress of Bayesian Neural Networks on
three different data-sets, including occlusion, support and quantity comparison
tasks. We compare the results with prior research from developmental psychology
and find agreement between the family of optimized models and pattern of
development observed in infants and children on all three tasks, indicating
common principles for the acquisition of knowledge.
Active Probabilistic Inference on Matrices for Pre-Conditioning in Stochastic Optimization
Authors: Filip de Roos, Philipp Hennig
Date: 2019-02-20 00:00:00
Tasks: Stochastic Optimization

Pre-conditioning is a well-known concept that can significantly improve the
convergence of optimization algorithms. For noise-free problems, where good
pre-conditioners are not known a priori, iterative linear algebra methods offer
one way to efficiently construct them. For the stochastic optimization problems
that dominate contemporary machine learning, however, this approach is not
readily available. We propose an iterative algorithm inspired by classic
iterative linear solvers that uses a probabilistic model to actively infer a
pre-conditioner in situations where Hessian-projections can only be constructed
with strong Gaussian noise. The algorithm is empirically demonstrated to
efficiently construct effective pre-conditioners for stochastic gradient
descent and its variants. Experiments on problems of comparably low
dimensionality show improved convergence. In very high-dimensional problems,
such as those encountered in deep learning, the pre-conditioner effectively
becomes an automatic learning-rate adaptation scheme, which we also empirically
show to work well.
Data collaboration analysis for distributed datasets
Authors: Akira Imakura, Tetsuya Sakurai
Date: 2019-02-20 00:00:00
Tasks: Privacy Preserving

In this paper, we propose a data collaboration analysis method for
distributed datasets. The proposed method is a centralized machine learning
while training datasets and models remain distributed over some institutions.
Recently, data became large and distributed with decreasing costs of data
collection. If we can centralize these distributed datasets and analyse them as
one dataset, we expect to obtain novel insight and achieve a higher prediction
performance compared with individual analyses on each distributed dataset.
However, it is generally difficult to centralize the original datasets due to
their huge data size or regarding a privacy-preserving problem. To avoid these
difficulties, we propose a data collaboration analysis method for distributed
datasets without sharing the original datasets. The proposed method centralizes
only intermediate representation constructed individually instead of the
original dataset.
Convergence of Learning Dynamics in Information Retrieval Games
Authors: Omer Ben-Porat, Itay Rosenberg, Moshe Tennenholtz
Date: 2018-06-14 00:00:00
Tasks: Information Retrieval, Retrieval

We consider a game-theoretic model of information retrieval with strategic
authors. We examine two different utility schemes: authors who aim at
maximizing exposure and authors who want to maximize active selection of their
content (i.e. the number of clicks). We introduce the study of author learning
dynamics in such contexts. We prove that under the probability ranking
principle (PRP), which forms the basis of the current state of the art ranking
methods, any better-response learning dynamics converges to a pure Nash
equilibrium. We also show that other ranking methods induce a strategic
environment under which such a convergence may not occur.
Adaptive scale-invariant online algorithms for learning linear models
Authors: Michał Kempka, Wojciech Kotłowski, Manfred K. Warmuth
Date: 2019-02-20 00:00:00

We consider online learning with linear models, where the algorithm predicts
on sequentially revealed instances (feature vectors), and is compared against
the best linear function (comparator) in hindsight. Popular algorithms in this
framework, such as Online Gradient Descent (OGD), have parameters (learning
rates), which ideally should be tuned based on the scales of the features and
the optimal comparator, but these quantities only become available at the end
of the learning process. In this paper, we resolve the tuning problem by
proposing online algorithms making predictions which are invariant under
arbitrary rescaling of the features. The algorithms have no parameters to tune,
do not require any prior knowledge on the scale of the instances or the
comparator, and achieve regret bounds matching (up to a logarithmic factor)
that of OGD with optimally tuned separate learning rates per dimension, while
retaining comparable runtime performance.
Resolving Conflicts in Clinical Guidelines using Argumentation
Authors: Kristijonas Čyras, Tiago Oliveira
Date: 2019-02-20 00:00:00

Automatically reasoning with conflicting generic clinical guidelines is a
burning issue in patient-centric medical reasoning where patient-specific
conditions and goals need to be taken into account. It is even more challenging
in the presence of preferences such as patient's wishes and clinician's
priorities over goals. We advance a structured argumentation formalism for
reasoning with conflicting clinical guidelines, patient-specific information
and preferences. Our formalism integrates assumption-based reasoning and
goal-driven selection among reasoning outcomes. Specifically, we assume
applicability of guideline recommendations concerning the generic goal of
patient well-being, resolve conflicts among recommendations using patient's
conditions and preferences, and then consider prioritised patient-centered
goals to yield non-conflicting, goal-maximising and preference-respecting
recommendations. We rely on the state-of-the-art Transition-based Medical
Recommendation model for representing guideline recommendations and augment it
with context given by the patient's conditions, goals, as well as preferences
over recommendations and goals. We establish desirable properties of our
approach in terms of sensitivity to recommendation conflicts and patient
context.