pwc-archive/papers-with-abstracts huggingface dataset:
GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objects
Authors: Edward J. Smith, Scott Fujimoto, Adriana Romero, David Meger
Date: 2019-01-31 00:00:00
Tasks: 3D Object Reconstruction, Object Reconstruction

Mesh models are a promising approach for encoding the structure of 3D
objects. Current mesh reconstruction systems predict uniformly distributed
vertex locations of a predetermined graph through a series of graph
convolutions, leading to compromises with respect to performance or resolution.
In this paper, we argue that the graph representation of geometric objects
allows for additional structure, which should be leveraged for enhanced
reconstruction. Thus, we propose a system which properly benefits from the
advantages of the geometric structure of graph encoded objects by introducing
(1) a graph convolutional update preserving vertex information; (2) an adaptive
splitting heuristic allowing detail to emerge; and (3) a training objective
operating both on the local surfaces defined by vertices as well as the global
structure defined by the mesh. Our proposed method is evaluated on the task of
3D object reconstruction from images with the ShapeNet dataset, where we
demonstrate state of the art performance, both visually and numerically, while
having far smaller space requirements by generating adaptive meshes
Funnelling: A New Ensemble Method for Heterogeneous Transfer Learning and its Application to Cross-Lingual Text Classification
Authors: Andrea Esuli, Alejandro Moreo, Fabrizio Sebastiani
Date: 2019-01-31 00:00:00
Tasks: All, Classification, Ensemble Learning, General Classification, text-classification, Text Classification, Transfer Learning

Cross-lingual Text Classification (CLC) consists of automatically
classifying, according to a common set C of classes, documents each written in
one of a set of languages L, and doing so more accurately than when naively
classifying each document via its corresponding language-specific classifier.
In order to obtain an increase in the classification accuracy for a given
language, the system thus needs to also leverage the training examples written
in the other languages. We tackle multilabel CLC via funnelling, a new ensemble
learning method that we propose here. Funnelling consists of generating a
two-tier classification system where all documents, irrespectively of language,
are classified by the same (2nd-tier) classifier. For this classifier all
documents are represented in a common, language-independent feature space
consisting of the posterior probabilities generated by 1st-tier,
language-dependent classifiers. This allows the classification of all test
documents, of any language, to benefit from the information present in all
training documents, of any language. We present substantial experiments, run on
publicly available multilingual text collections, in which funnelling is shown
to significantly outperform a number of state-of-the-art baselines. All code
and datasets (in vector form) are made publicly available.
Effect of Various Regularizers on Model Complexities of Neural Networks in Presence of Input Noise
Authors: Mayank Sharma, Aayush Yadav, Sumit Soman, Jayadeva
Date: 2019-01-31 00:00:00

Deep neural networks are over-parameterized, which implies that the number of
parameters are much larger than the number of samples used to train the
network. Even in such a regime deep architectures do not overfit. This
phenomenon is an active area of research and many theories have been proposed
trying to understand this peculiar observation. These include the Vapnik
Chervonenkis (VC) dimension bounds and Rademacher complexity bounds which show
that the capacity of the network is characterized by the norm of weights rather
than the number of parameters. However, the effect of input noise on these
measures for shallow and deep architectures has not been studied. In this
paper, we analyze the effects of various regularization schemes on the
complexity of a neural network which we characterize with the loss, $L_2$ norm
of the weights, Rademacher complexities (Directly Approximately Regularizing
Complexity-DARC1), VC dimension based Low Complexity Neural Network (LCNN) when
subject to varying degrees of Gaussian input noise. We show that $L_2$
regularization leads to a simpler hypothesis class and better generalization
followed by DARC1 regularizer, both for shallow as well as deeper
architectures. Jacobian regularizer works well for shallow architectures with
high level of input noises. Spectral normalization attains highest test set
accuracies both for shallow and deeper architectures. We also show that Dropout
alone does not perform well in presence of input noise. Finally, we show that
deeper architectures are robust to input noise as opposed to their shallow
counterparts.
Improving SGD convergence by online linear regression of gradients in multiple statistically relevant directions
Authors: Jarek Duda
Date: 2019-01-31 00:00:00
Tasks: regression, Second-order methods

Deep neural networks are usually trained with stochastic gradient descent (SGD), which minimizes objective function using very rough approximations of gradient, only averaging to the real gradient. Standard approaches like momentum or ADAM only consider a single direction, and do not try to model distance from extremum - neglecting valuable information from calculated sequence of gradients, often stagnating in some suboptimal plateau. Second order methods could exploit these missed opportunities, however, beside suffering from very large cost and numerical instabilities, many of them attract to suboptimal points like saddles due to negligence of signs of curvatures (as eigenvalues of Hessian). Saddle-free Newton method is a rare example of addressing this issue - changes saddle attraction into repulsion, and was shown to provide essential improvement for final value this way. However, it neglects noise while modelling second order behavior, focuses on Krylov subspace for numerical reasons, and requires costly eigendecomposion. Maintaining SFN advantages, there are proposed inexpensive ways for exploiting these opportunities. Second order behavior is linear dependence of first derivative - we can optimally estimate it from sequence of noisy gradients with least square linear regression, in online setting here: with weakening weights of old gradients. Statistically relevant subspace is suggested by PCA of recent noisy gradients - in online setting it can be made by slowly rotating considered directions toward new gradients, gradually replacing old directions with recent statistically relevant. Eigendecomposition can be also performed online: with regularly performed step of QR method to maintain diagonal Hessian. Outside the second order modeled subspace we can simultaneously perform gradient descent.
Feature-Critic Networks for Heterogeneous Domain Generalization
Authors: Yiying Li, Yongxin Yang, Wei Zhou, Timothy M. Hospedales
Date: 2019-01-31 00:00:00
Tasks: Domain Adaptation, Domain Generalization

The well known domain shift issue causes model performance to degrade when deployed to a new target domain with different statistics to training. Domain adaptation techniques alleviate this, but need some instances from the target domain to drive adaptation. Domain generalisation is the recently topical problem of learning a model that generalises to unseen domains out of the box, and various approaches aim to train a domain-invariant feature extractor, typically by adding some manually designed losses. In this work, we propose a learning to learn approach, where the auxiliary loss that helps generalisation is itself learned. Beyond conventional domain generalisation, we consider a more challenging setting of heterogeneous domain generalisation, where the unseen domains do not share label space with the seen ones, and the goal is to train a feature representation that is useful off-the-shelf for novel data and novel categories. Experimental evaluation demonstrates that our method outperforms state-of-the-art solutions in both settings.
Automated detection of celiac disease on duodenal biopsy slides: a deep learning approach
Authors: Jason W. Wei, Jerry W. Wei, Christopher R. Jackson, Bing Ren, Arief A. Suriawinata, Saeed Hassanpour
Date: 2019-01-31 00:00:00
Tasks: Classification, General Classification

Celiac disease prevalence and diagnosis have increased substantially in
recent years. The current gold standard for celiac disease confirmation is
visual examination of duodenal mucosal biopsies. An accurate computer-aided
biopsy analysis system using deep learning can help pathologists diagnose
celiac disease more efficiently. In this study, we trained a deep learning
model to detect celiac disease on duodenal biopsy images. Our model uses a
state-of-the-art residual convolutional neural network to evaluate patches of
duodenal tissue and then aggregates those predictions for whole-slide
classification. We tested the model on an independent set of 212 images and
evaluated its classification results against reference standards established by
pathologists. Our model identified celiac disease, normal tissue, and
nonspecific duodenitis with accuracies of 95.3%, 91.0%, and 89.2%,
respectively. The area under the receiver operating characteristic curve was
greater than 0.95 for all classes. We have developed an automated biopsy
analysis system that achieves high performance in detecting celiac disease on
biopsy slides. Our system can highlight areas of interest and provide
preliminary classification of duodenal biopsies prior to review by
pathologists. This technology has great potential for improving the accuracy
and efficiency of celiac disease diagnosis.
Toward Sensor-based Sleep Monitoring with Electrodermal Activity Measures
Authors: William Romine, Tanvi Banerjee, Garrett Goodman
Date: 2019-01-31 00:00:00
Tasks: Sleep Quality

We use self-report and electrodermal activity (EDA) wearable sensor data from
77 nights of sleep on six participants to test the efficacy of EDA data for
sleep monitoring. We used factor analysis to find latent factors in the EDA
data, and causal model search to find the most probable graphical model
accounting for self-reported sleep efficiency (SE), sleep quality (SQ), and the
latent EDA factors. Structural equation modeling was used to confirm fit of the
extracted graph. Based on the generated graph, logistic regression and naive
Bayes models were used to test the efficacy of the EDA data in predicting SE
and SQ. Six EDA features extracted from the total signal over a night's sleep
could be explained by two latent factors, EDA Magnitude and EDA Storms. EDA
Magnitude performed as a strong predictor for SE to aid detection of
substantial changes in time asleep. The performance of EDA Magnitured and SE in
classifying SQ showed promise for wearable sleep monitoring applications.
However, our data suggest that obtaining a more accurate sensor-based measure
of SE will be necessary before smaller changes in SQ can be detected from EDA
sensor data alone.
Successor Features Combine Elements of Model-Free and Model-based Reinforcement Learning
Authors: Lucas Lehnert, Michael L. Littman
Date: 2019-01-31 00:00:00
Tasks: Decision Making, model, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A key question in reinforcement learning is how an intelligent agent can generalize knowledge across different inputs. By generalizing across different inputs, information learned for one input can be immediately reused for improving predictions for another input. Reusing information allows an agent to compute an optimal decision-making strategy using less data. State representation is a key element of the generalization process, compressing a high-dimensional input space into a low-dimensional latent state space. This article analyzes properties of different latent state spaces, leading to new connections between model-based and model-free reinforcement learning. Successor features, which predict frequencies of future observations, form a link between model-based and model-free learning: Learning to predict future expected reward outcomes, a key characteristic of model-based agents, is equivalent to learning successor features. Learning successor features is a form of temporal difference learning and is equivalent to learning to predict a single policy's utility, which is a characteristic of model-free agents. Drawing on the connection between model-based reinforcement learning and successor features, we demonstrate that representations that are predictive of future reward outcomes generalize across variations in both transitions and rewards. This result extends previous work on successor features, which is constrained to fixed transitions and assumes re-learning of the transferred state representation.
End-to-End Probabilistic Inference for Nonstationary Audio Analysis
Authors: William J. Wilkinson, Michael Riis Andersen, Joshua D. Reiss, Dan Stowell, Arno Solin
Date: 2019-01-31 00:00:00
Tasks: Audio Signal Processing, regression

A typical audio signal processing pipeline includes multiple disjoint
analysis stages, including calculation of a time-frequency representation
followed by spectrogram-based feature analysis. We show how time-frequency
analysis and nonnegative matrix factorisation can be jointly formulated as a
spectral mixture Gaussian process model with nonstationary priors over the
amplitude variance parameters. Further, we formulate this nonlinear model's
state space representation, making it amenable to infinite-horizon Gaussian
process regression with approximate inference via expectation propagation,
which scales linearly in the number of time steps and quadratically in the
state dimensionality. By doing so, we are able to process audio signals with
hundreds of thousands of data points. We demonstrate, on various tasks with
empirical data, how this inference scheme outperforms more standard techniques
that rely on extended Kalman filtering.
Input Redundancy for Parameterized Quantum Circuits
Authors: Javier Gil Vidal, Dirk Oliver Theis
Date: 2019-01-31 00:00:00

The topic area of this paper parameterized quantum circuits (quantum neural networks) which are trained to estimate a given function, specifically the type of circuits proposed by Mitarai et al. (Phys. Rev. A, 2018). The input is encoded into amplitudes of states of qubits. The no-cloning principle of quantum mechanics suggests that there is an advantage in redundantly encoding the input value several times. We follow this suggestion and prove lower bounds on the number of redundant copies for two types of input encoding. We draw conclusions for the architecture design of QNNs.
Decomposing Generalization: Models of Generic, Habitual, and Episodic Statements
Authors: Venkata Subrahmanyan Govindarajan, Benjamin Van Durme, Aaron Steven White
Date: 2019-01-31 00:00:00
Proceeding: TACL 2019 3
Tasks: Word Embeddings

We present a novel semantic framework for modeling linguistic expressions of generalization---generic, habitual, and episodic statements---as combinations of simple, real-valued referential properties of predicates and their arguments. We use this framework to construct a dataset covering the entirety of the Universal Dependencies English Web Treebank. We use this dataset to probe the efficacy of type-level and token-level information---including hand-engineered features and static (GloVe) and contextual (ELMo) word embeddings---for predicting expressions of generalization. Data and code are available at decomp.io.
High-dimensional Metric Combining for Non-coherent Molecular Signal Detection
Authors: Zhuangkun Wei, Weisi Guo, Bin Li, Jerome Charmet, Chenglin Zhao
Date: 2019-01-31 00:00:00
Tasks: Vocal Bursts Intensity Prediction

In emerging Internet-of-Nano-Thing (IoNT), information will be embedded and
conveyed in the form of molecules through complex and diffusive medias. One
main challenge lies in the long-tail nature of the channel response causing
inter-symbol-interference (ISI), which deteriorates the detection performance.
If the channel is unknown, we cannot easily achieve traditional coherent
channel estimation and cancellation, and the impact of ISI will be more severe.
In this paper, we develop a novel high-dimensional non-coherent scheme for
blind detection of molecular signals. We achieve this in a higher-dimensional
metric space by combining different non-coherent metrics that exploit the
transient features of the signals. By deducing the theoretical bit error rate
(BER) for any constructed high-dimensional non-coherent metric, we prove that,
higher dimensionality always achieves a lower BER in the same sample space.
Then, we design a generalised blind detection algorithm that utilizes the
Parzen approximation and its probabilistic neural network (Parzen-PNN) to
detect information bits. Taking advantages of its fast convergence and parallel
implementation, our proposed scheme can meet the needs of detection accuracy
and real-time computing. Numerical simulations demonstrate that our proposed
scheme can gain 10dB BER compared with other state of the art methods.
Is Image Memorability Prediction Solved?
Authors: Shay Perera, Ayellet Tal, Lihi Zelnik-Manor
Date: 2019-01-31 00:00:00
Tasks: Prediction

This paper deals with the prediction of the memorability of a given image. We
start by proposing an algorithm that reaches human-level performance on the
LaMem dataset - the only large scale benchmark for memorability prediction. The
suggested algorithm is based on three observations we make regarding
convolutional neural networks (CNNs) that affect memorability prediction.
Having reached human-level performance we were humbled, and asked ourselves
whether indeed we have resolved memorability prediction - and answered this
question in the negative. We studied a few factors and made some
recommendations that should be taken into account when designing the next
benchmark.
Sequential Bayesian Detection of Spike Activities from Fluorescence Observations
Authors: Zhuangkun Wei, Bin Li, Weisi Guo, Wenxiu Hu, Chenglin Zhao
Date: 2019-01-31 00:00:00
Tasks: Bayesian Inference

Extracting and detecting spike activities from the fluorescence observations
is an important step in understanding how neuron systems work. The main
challenge lies in that the combination of the ambient noise with dynamic
baseline fluctuation, often contaminates the observations, thereby
deteriorating the reliability of spike detection. This may be even worse in the
face of the nonlinear biological process, the coupling interactions between
spikes and baseline, and the unknown critical parameters of an underlying
physiological model, in which erroneous estimations of parameters will affect
the detection of spikes causing further error propagation. In this paper, we
propose a random finite set (RFS) based Bayesian approach. The dynamic
behaviors of spike sequence, fluctuated baseline and unknown parameters are
formulated as one RFS. This RFS state is capable of distinguishing the hidden
active/silent states induced by spike and non-spike activities respectively,
thereby \emph{negating the interaction role} played by spikes and other
factors. Then, premised on the RFS states, a Bayesian inference scheme is
designed to simultaneously estimate the model parameters, baseline, and crucial
spike activities. Our results demonstrate that the proposed scheme can gain an
extra $12\%$ detection accuracy in comparison with the state-of-the-art MLSpike
method.
Geometric fluid approximation for general continuous-time Markov chains
Authors: Michalis Michaelides, Jane Hillston, Guido Sanguinetti
Date: 2019-01-31 00:00:00

Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular population structure which suggests a natural continuous state-space endowed with a dynamics for the approximating process. We construct here a general method based on spectral analysis of the transition matrix of the CTMC, without the need for a population structure. Specifically, we use the popular manifold learning method of diffusion maps to analyse the transition matrix as the operator of a hidden continuous process. An embedding of states in a continuous space is recovered, and the space is endowed with a drift vector field inferred via Gaussian process regression. In this manner, we construct an ODE whose solution approximates the evolution of the CTMC mean, mapped onto the continuous space (known as the fluid limit).
Learning and Evaluating General Linguistic Intelligence
Authors: Dani Yogatama, Cyprien de Masson d'Autume, Jerome Connor, Tomas Kocisky, Mike Chrzanowski, Lingpeng Kong, Angeliki Lazaridou, Wang Ling, Lei Yu, Chris Dyer, Phil Blunsom
Date: 2019-01-31 00:00:00
Tasks: Natural Language Understanding, Question Answering

We define general linguistic intelligence as the ability to reuse previously
acquired knowledge about a language's lexicon, syntax, semantics, and pragmatic
conventions to adapt to new tasks quickly. Using this definition, we analyze
state-of-the-art natural language understanding models and conduct an extensive
empirical investigation to evaluate them against these criteria through a
series of experiments that assess the task-independence of the knowledge being
acquired by the learning process. In addition to task performance, we propose a
new evaluation metric based on an online encoding of the test data that
quantifies how quickly an existing agent (model) learns a new task. Our results
show that while the field has made impressive progress in terms of model
architectures that generalize to many tasks, these models still require a lot
of in-domain training examples (e.g., for fine tuning, training task-specific
modules), and are prone to catastrophic forgetting. Moreover, we find that far
from solving general tasks (e.g., document question answering), our models are
overfitting to the quirks of particular datasets (e.g., SQuAD). We discuss
missing components and conjecture on how to make progress toward general
linguistic intelligence.