pwc-archive/papers-with-abstracts huggingface dataset:
Deep Mixtures of Unigrams for uncovering Topics in Textual Data
Authors: Cinzia Viroli, Laura Anderlucci
Date: 2019-02-18 00:00:00
Tasks: Classification, Clustering, General Classification

Mixtures of Unigrams are one of the simplest and most efficient tools for clustering textual data, as they assume that documents related to the same topic have similar distributions of terms, naturally described by Multinomials. When the classification task is particularly challenging, such as when the document-term matrix is high-dimensional and extremely sparse, a more composite representation can provide better insight on the grouping structure. In this work, we developed a deep version of mixtures of Unigrams for the unsupervised classification of very short documents with a large number of terms, by allowing for models with further deeper latent layers; the proposal is derived in a Bayesian framework. The behaviour of the Deep Mixtures of Unigrams is empirically compared with that of other traditional and state-of-the-art methods, namely $k$-means with cosine distance, $k$-means with Euclidean distance on data transformed according to Semantic Analysis, Partition Around Medoids, Mixture of Gaussians on semantic-based transformed data, hierarchical clustering according to Ward's method with cosine dissimilarity, Latent Dirichlet Allocation, Mixtures of Unigrams estimated via the EM algorithm, Spectral Clustering and Affinity Propagation clustering. The performance is evaluated in terms of both correct classification rate and Adjusted Rand Index. Simulation studies and real data analysis prove that going deep in clustering such data highly improves the classification accuracy.
Object Recognition under Multifarious Conditions: A Reliability Analysis and A Feature Similarity-based Performance Estimation
Authors: Dogancan Temel, Jinsol Lee, Ghassan AlRegib
Date: 2019-02-18 00:00:00
Tasks: Object Recognition

In this paper, we investigate the reliability of online recognition platforms, Amazon Rekognition and Microsoft Azure, with respect to changes in background, acquisition device, and object orientation. We focus on platforms that are commonly used by the public to better understand their real-world performances. To assess the variation in recognition performance, we perform a controlled experiment by changing the acquisition conditions one at a time. We use three smartphones, one DSLR, and one webcam to capture side views and overhead views of objects in a living room, an office, and photo studio setups. Moreover, we introduce a framework to estimate the recognition performance with respect to backgrounds and orientations. In this framework, we utilize both handcrafted features based on color, texture, and shape characteristics and data-driven features obtained from deep neural networks. Experimental results show that deep learning-based image representations can estimate the recognition performance variation with a Spearman's rank-order correlation of 0.94 under multifarious acquisition conditions.
Fast Efficient Hyperparameter Tuning for Policy Gradients
Authors: Supratik Paul, Vitaly Kurin, Shimon Whiteson
Date: 2019-02-18 00:00:00
Tasks: Meta-Learning, Policy Gradient Methods

The performance of policy gradient methods is sensitive to hyperparameter settings that must be tuned for any new application. Widely used grid search methods for tuning hyperparameters are sample inefficient and computationally expensive. More advanced methods like Population Based Training that learn optimal schedules for hyperparameters instead of fixed settings can yield better results, but are also sample inefficient and computationally expensive. In this paper, we propose Hyperparameter Optimisation on the Fly (HOOF), a gradient-free algorithm that requires no more than one training run to automatically adapt the hyperparameter that affect the policy update directly through the gradient. The main idea is to use existing trajectories sampled by the policy gradient method to optimise a one-step improvement objective, yielding a sample and computationally efficient algorithm that is easy to implement. Our experimental results across multiple domains and algorithms show that using HOOF to learn these hyperparameter schedules leads to faster learning with improved performance.
Conformal calibrators
Authors: Vladimir Vovk, Ivan Petej, Paolo Toccaceli, Alex Gammerman
Date: 2019-02-18 00:00:00
Tasks: Conformal Prediction

Most existing examples of full conformal predictive systems, split-conformal
predictive systems, and cross-conformal predictive systems impose severe
restrictions on the adaptation of predictive distributions to the test object
at hand. In this paper we develop split-conformal and cross-conformal
predictive systems that are fully adaptive. Our method consists in calibrating
existing predictive systems; the input predictive system is not supposed to
satisfy any properties of validity, whereas the output predictive system is
guaranteed to be calibrated in probability. It is interesting that the method
may also work without the IID assumption, standard in conformal prediction.
STCN: Stochastic Temporal Convolutional Networks
Authors: Emre Aksan, Otmar Hilliges
Date: 2019-02-18 00:00:00
Proceeding: ICLR 2019 5

Convolutional architectures have recently been shown to be competitive on
many sequence modelling tasks when compared to the de-facto standard of
recurrent neural networks (RNNs), while providing computational and modeling
advantages due to inherent parallelism. However, currently there remains a
performance gap to more expressive stochastic RNN variants, especially those
with several layers of dependent random variables. In this work, we propose
stochastic temporal convolutional networks (STCNs), a novel architecture that
combines the computational advantages of temporal convolutional networks (TCN)
with the representational power and robustness of stochastic latent spaces. In
particular, we propose a hierarchy of stochastic latent variables that captures
temporal dependencies at different time-scales. The architecture is modular and
flexible due to the decoupling of the deterministic and stochastic layers. We
show that the proposed architecture achieves state of the art log-likelihoods
across several tasks. Finally, the model is capable of predicting high-quality
synthetic samples over a long-range temporal horizon in modeling of handwritten
text.
The Kalai-Smorodinski solution for many-objective Bayesian optimization
Authors: Mickaël Binois, Victor Picheny, Patrick Taillandier, Abderrahmane Habbal
Date: 2019-02-18 00:00:00
Tasks: Bayesian Optimization

An ongoing aim of research in multiobjective Bayesian optimization is to extend its applicability to a large number of objectives. While coping with a limited budget of evaluations, recovering the set of optimal compromise solutions generally requires numerous observations and is less interpretable since this set tends to grow larger with the number of objectives. We thus propose to focus on a specific solution originating from game theory, the Kalai-Smorodinsky solution, which possesses attractive properties. In particular, it ensures equal marginal gains over all objectives. We further make it insensitive to a monotonic transformation of the objectives by considering the objectives in the copula space. A novel tailored algorithm is proposed to search for the solution, in the form of a Bayesian optimization algorithm: sequential sampling decisions are made based on acquisition functions that derive from an instrumental Gaussian process prior. Our approach is tested on four problems with respectively four, six, eight, and nine objectives. The method is available in the Rpackage GPGame available on CRAN at https://cran.r-project.org/package=GPGame.
Intra- and Inter-epoch Temporal Context Network (IITNet) Using Sub-epoch Features for Automatic Sleep Scoring on Raw Single-channel EEG
Authors: Hogeon Seo, Seunghyeok Back, Seongju Lee, Deokhwan Park, Tae Kim, Kyoobin Lee
Date: 2019-02-18 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Sleep Stage Detection

A deep learning model, named IITNet, is proposed to learn intra- and inter-epoch temporal contexts from raw single-channel EEG for automatic sleep scoring. To classify the sleep stage from half-minute EEG, called an epoch, sleep experts investigate sleep-related events and consider the transition rules between the found events. Similarly, IITNet extracts representative features at a sub-epoch level by a residual neural network and captures intra- and inter-epoch temporal contexts from the sequence of the features via bidirectional LSTM. The performance was investigated for three datasets as the sequence length (L) increased from one to ten. IITNet achieved the comparable performance with other state-of-the-art results. The best accuracy, MF1, and Cohen's kappa ($\kappa$) were 83.9%, 77.6%, 0.78 for SleepEDF (L=10), 86.5%, 80.7%, 0.80 for MASS (L=9), and 86.7%, 79.8%, 0.81 for SHHS (L=10), respectively. Even though using four epochs, the performance was still comparable. Compared to using a single epoch, on average, accuracy and MF1 increased by 2.48%p and 4.90%p and F1 of N1, N2, and REM increased by 16.1%p, 1.50%p, and 6.42%p, respectively. Above four epochs, the performance improvement was not significant. The results support that considering the latest two-minute raw single-channel EEG can be a reasonable choice for sleep scoring via deep neural networks with efficiency and reliability. Furthermore, the experiments with the baselines showed that introducing intra-epoch temporal context learning with a deep residual network contributes to the improvement in the overall performance and has the positive synergy effect with the inter-epoch temporal context learning.
Decomposing multispectral face images into diffuse and specular shading and biophysical parameters
Authors: Sarah Alotaibi, William A. P. Smith
Date: 2019-02-18 00:00:00

We propose a novel biophysical and dichromatic reflectance model that
efficiently characterises spectral skin reflectance. We show how to fit the
model to multispectral face images enabling high quality estimation of diffuse
and specular shading as well as biophysical parameter maps (melanin and
haemoglobin). Our method works from a single image without requiring complex
controlled lighting setups yet provides quantitatively accurate reconstructions
and qualitatively convincing decomposition and editing.
MetaGrasp: Data Efficient Grasping by Affordance Interpreter Network
Authors: Junhao Cai, Hui Cheng, Zhanpeng Zhang, Jingcheng Su
Date: 2019-02-18 00:00:00

Data-driven approach for grasping shows significant advance recently. But
these approaches usually require much training data. To increase the efficiency
of grasping data collection, this paper presents a novel grasp training system
including the whole pipeline from data collection to model inference. The
system can collect effective grasp sample with a corrective strategy assisted
by antipodal grasp rule, and we design an affordance interpreter network to
predict pixelwise grasp affordance map. We define graspability, ungraspability
and background as grasp affordances. The key advantage of our system is that
the pixel-level affordance interpreter network trained with only a small number
of grasp samples under antipodal rule can achieve significant performance on
totally unseen objects and backgrounds. The training sample is only collected
in simulation. Extensive qualitative and quantitative experiments demonstrate
the accuracy and robustness of our proposed approach. In the real-world grasp
experiments, we achieve a grasp success rate of 93% on a set of household items
and 91% on a set of adversarial items with only about 6,300 simulated samples.
We also achieve 87% accuracy in clutter scenario. Although the model is trained
using only RGB image, when changing the background textures, it also performs
well and can achieve even 94% accuracy on the set of adversarial objects, which
outperforms current state-of-the-art methods.
A computational model for grid maps in neural populations
Authors: Fabio Anselmi, Micah M. Murray, Benedetta Franceschiello
Date: 2019-02-18 00:00:00

Grid cells in the entorhinal cortex, together with head direction, place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this work we introduce a novel theoretical and algorithmic framework able to explain the emergence of hexagonal grid-like response patterns from head direction cells' responses. We show that this pattern is a result of minimal variance encoding of neurons. The novelty lies into the formulation of the encoding problem through the modern Frame Theory language, specifically that of equiangular Frames, providing new insights about the optimality of hexagonal grid receptive fields. The model proposed overcomes some crucial limitations of the current attractor and oscillatory models. It is based on the well-accepted and tested hypothesis of Hebbian learning, providing a simplified cortical-based framework that does not require the presence of theta velocity-driven oscillations (oscillatory model) or translational symmetries in the synaptic connections (attractor model). We moreover demonstrate that the proposed encoding mechanism naturally explains axis alignment of neighbor grid cells and maps shifts, rotations and scaling of the stimuli onto the shape of grid cells' receptive fields, giving a straightforward explanation of the experimental evidence of grid cells remapping under transformations of environmental cues.
LocalNorm: Robust Image Classification through Dynamically Regularized Normalization
Authors: Bojian Yin, Siebren Schaafsma, Henk Corporaal, H. Steven Scholte, Sander M. Bohte
Date: 2019-02-18 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

While modern convolutional neural networks achieve outstanding accuracy on
many image classification tasks, they are, compared to humans, much more
sensitive to image degradation. Here, we describe a variant of Batch
Normalization, LocalNorm, that regularizes the normalization layer in the
spirit of Dropout while dynamically adapting to the local image intensity and
contrast at test-time. We show that the resulting deep neural networks are much
more resistant to noise-induced image degradation, improving accuracy by up to
three times, while achieving the same or slightly better accuracy on
non-degraded classical benchmarks. In computational terms, LocalNorm adds
negligible training cost and little or no cost at inference time, and can be
applied to already-trained networks in a straightforward manner.
Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology
Authors: David Tellez, Geert Litjens, Peter Bandi, Wouter Bulten, John-Melle Bokhorst, Francesco Ciompi, Jeroen van der Laak
Date: 2019-02-18 00:00:00
Tasks: Color Normalization, Data Augmentation

Stain variation is a phenomenon observed when distinct pathology laboratories stain tissue slides that exhibit similar but not identical color appearance. Due to this color shift between laboratories, convolutional neural networks (CNNs) trained with images from one lab often underperform on unseen images from the other lab. Several techniques have been proposed to reduce the generalization error, mainly grouped into two categories: stain color augmentation and stain color normalization. The former simulates a wide variety of realistic stain variations during training, producing stain-invariant CNNs. The latter aims to match training and test color distributions in order to reduce stain variation. For the first time, we compared some of these techniques and quantified their effect on CNN classification performance using a heterogeneous dataset of hematoxylin and eosin histopathology images from 4 organs and 9 pathology laboratories. Additionally, we propose a novel unsupervised method to perform stain color normalization using a neural network. Based on our experimental results, we provide practical guidelines on how to use stain color augmentation and stain color normalization in future computational pathology applications.
Optimizing Stochastic Gradient Descent in Text Classification Based on Fine-Tuning Hyper-Parameters Approach. A Case Study on Automatic Classification of Global Terrorist Attacks
Authors: Shadi Diab
Date: 2019-02-18 00:00:00
Tasks: Classification, General Classification, text-classification, Text Classification

The objective of this research is to enhance performance of Stochastic
Gradient Descent (SGD) algorithm in text classification. In our research, we
proposed using SGD learning with Grid-Search approach to fine-tuning
hyper-parameters in order to enhance the performance of SGD classification. We
explored different settings for representation, transformation and weighting
features from the summary description of terrorist attacks incidents obtained
from the Global Terrorism Database as a pre-classification step, and validated
SGD learning on Support Vector Machine (SVM), Logistic Regression and
Perceptron classifiers by stratified 10-K-fold cross-validation to compare the
performance of different classifiers embedded in SGD algorithm. The research
concludes that using a grid-search to find the hyper-parameters optimize SGD
classification, not in the pre-classification settings only, but also in the
performance of the classifiers in terms of accuracy and execution time.
Message-Dropout: An Efficient Training Method for Multi-Agent Deep Reinforcement Learning
Authors: Woojun Kim, Myungsik Cho, Youngchul Sung
Date: 2019-02-18 00:00:00
Tasks: Deep Reinforcement Learning, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this paper, we propose a new learning technique named message-dropout to
improve the performance for multi-agent deep reinforcement learning under two
application scenarios: 1) classical multi-agent reinforcement learning with
direct message communication among agents and 2) centralized training with
decentralized execution. In the first application scenario of multi-agent
systems in which direct message communication among agents is allowed, the
message-dropout technique drops out the received messages from other agents in
a block-wise manner with a certain probability in the training phase and
compensates for this effect by multiplying the weights of the dropped-out block
units with a correction probability. The applied message-dropout technique
effectively handles the increased input dimension in multi-agent reinforcement
learning with communication and makes learning robust against communication
errors in the execution phase. In the second application scenario of
centralized training with decentralized execution, we particularly consider the
application of the proposed message-dropout to Multi-Agent Deep Deterministic
Policy Gradient (MADDPG), which uses a centralized critic to train a
decentralized actor for each agent. We evaluate the proposed message-dropout
technique for several games, and numerical results show that the proposed
message-dropout technique with proper dropout rate improves the reinforcement
learning performance significantly in terms of the training speed and the
steady-state performance in the execution phase.
Prediction of Porosity and Permeability Alteration based on Machine Learning Algorithms
Authors: Andrei Erofeev, Denis Orlov, Alexey Ryzhov, Dmitry Koroteev
Date: 2019-02-18 00:00:00
Tasks: BIG-bench Machine Learning

The objective of this work is to study the applicability of various Machine
Learning algorithms for prediction of some rock properties which geoscientists
usually define due to special lab analysis. We demonstrate that these special
properties can be predicted only basing on routine core analysis (RCA) data. To
validate the approach core samples from the reservoir with soluble rock matrix
components (salts) were tested within 100+ laboratory experiments. The
challenge of the experiments was to characterize the rate of salts in cores and
alteration of porosity and permeability after reservoir desalination due to
drilling mud or water injection. For these three measured characteristics, we
developed the relevant predictive models, which were based on the results of
RCA and data on coring depth and top and bottom depths of productive horizons.
To select the most accurate Machine Learning algorithm a comparative analysis
has been performed. It was shown that different algorithms work better in
different models. However, two hidden layers Neural network has demonstrated
the best predictive ability and generalizability for all three rock
characteristics jointly. The other algorithms, such as Support Vector Machine
and Linear Regression, also worked well on the dataset, but in particular
cases. Overall, the applied approach allows predicting the alteration of
porosity and permeability during desalination in porous rocks and also
evaluating salt concentration without direct measurements in a laboratory. This
work also shows that developed approaches could be applied for prediction of
other rock properties (residual brine and oil saturations, relative
permeability, capillary pressure, and others), which laboratory measurements
are time-consuming and expensive.
Designing recurrent neural networks by unfolding an l1-l1 minimization algorithm
Authors: Hung Duy Le, Huynh Van Luong, Nikos Deligiannis
Date: 2019-02-18 00:00:00

We propose a new deep recurrent neural network (RNN) architecture for
sequential signal reconstruction. Our network is designed by unfolding the
iterations of the proximal gradient method that solves the l1-l1 minimization
problem. As such, our network leverages by design that signals have a sparse
representation and that the difference between consecutive signal
representations is also sparse. We evaluate the proposed model in the task of
reconstructing video frames from compressive measurements and show that it
outperforms several state-of-the-art RNN models.
Grids versus Graphs: Partitioning Space for Improved Taxi Demand-Supply Forecasts
Authors: Neema Davis, Gaurav Raina, Krishna Jagannathan
Date: 2019-02-18 00:00:00
Tasks: Ensemble Learning

Accurate taxi demand-supply forecasting is a challenging application of ITS
(Intelligent Transportation Systems), due to the complex spatial and temporal
patterns. We investigate the impact of different spatial partitioning
techniques on the prediction performance of an LSTM (Long Short-Term Memory)
network, in the context of taxi demand-supply forecasting. We consider two
tessellation schemes: (i) the variable-sized Voronoi tessellation, and (ii) the
fixed-sized Geohash tessellation. While the widely employed ConvLSTM
(Convolutional LSTM) can model fixed-sized Geohash partitions, the standard
convolutional filters cannot be applied on the variable-sized Voronoi
partitions. To explore the Voronoi tessellation scheme, we propose the use of
GraphLSTM (Graph-based LSTM), by representing the Voronoi spatial partitions as
nodes on an arbitrarily structured graph. The GraphLSTM offers competitive
performance against ConvLSTM, at lower computational complexity, across three
real-world large-scale taxi demand-supply data sets, with different performance
metrics. To ensure superior performance across diverse settings, a HEDGE based
ensemble learning algorithm is applied over the ConvLSTM and the GraphLSTM
networks.