pwc-archive/papers-with-abstracts huggingface dataset:
Episodic Training for Domain Generalization
Authors: Da Li, Jianshu Zhang, Yongxin Yang, Cong Liu, Yi-Zhe Song, Timothy M. Hospedales
Date: 2019-01-31 00:00:00
Proceeding: ICCV 2019 10
Tasks: Domain Generalization

Domain generalization (DG) is the challenging and topical problem of learning models that generalize to novel testing domains with different statistics than a set of known training domains. The simple approach of aggregating data from all source domains and training a single deep neural network end-to-end on all the data provides a surprisingly strong baseline that surpasses many prior published methods. In this paper, we build on this strong baseline by designing an episodic training procedure that trains a single deep network in a way that exposes it to the domain shift that characterises a novel domain at runtime. Specifically, we decompose a deep network into feature extractor and classifier components, and then train each component by simulating it interacting with a partner who is badly tuned for the current domain. This makes both components more robust, ultimately leading to our networks producing state-of-the-art performance on three DG benchmarks. Furthermore, we consider the pervasive workflow of using an ImageNet trained CNN as a fixed feature extractor for downstream recognition tasks. Using the Visual Decathlon benchmark, we demonstrate that our episodic-DG training improves the performance of such a general-purpose feature extractor by explicitly training a feature for robustness to novel problems. This shows that DG training can benefit standard practice in computer vision.
Parallel Black-Box Complexity with Tail Bounds
Authors: Per Kristian Lehre, Dirk Sudholt
Date: 2019-01-31 00:00:00
Tasks: Evolutionary Algorithms

We propose a new black-box complexity model for search algorithms evaluating
$\lambda$ search points in parallel. The parallel unary unbiased black-box
complexity gives lower bounds on the number of function evaluations every
parallel unary unbiased black-box algorithm needs to optimise a given problem.
It captures the inertia caused by offspring populations in evolutionary
algorithms and the total computational effort in parallel metaheuristics.
  We present complexity results for LeadingOnes and OneMax. Our main result is
a general performance limit: we prove that on every function every
$\lambda$-parallel unary unbiased algorithm needs at least
$\Omega(\frac{\lambda n}{\ln \lambda} + n \log n)$ evaluations to find any
desired target set of up to exponential size, with an overwhelming probability.
This yields lower bounds for the typical optimisation time on unimodal and
multimodal problems, for the time to find any local optimum, and for the time
to even get close to any optimum. The power and versatility of this approach is
shown for a wide range of illustrative problems from combinatorial
optimisation. Our performance limits can guide parameter choice and algorithm
design; we demonstrate the latter by presenting an optimal $\lambda$-parallel
algorithm for OneMax that uses parallelism most effectively.
On the statistical evaluation of algorithmic's computational experimentation with infeasible solutions
Authors: Iago A Carvalho
Date: 2019-01-31 00:00:00

The experimental evaluation of algorithms results in a large set of data
which generally do not follow a normal distribution or are not heteroscedastic.
Besides, some of its entries may be missing, due to the inability of an
algorithm to find a feasible solution until a time limit is met. Those
characteristics restrict the statistical evaluation of computational
experiments. This work proposes a bi-objective lexicographical ranking scheme
to evaluate datasets with such characteristics. The output ranking can be used
as input to any desired statistical test. We used the proposed ranking scheme
to assess the results obtained by the Iterative Rounding heuristic (IR). A
Friedman's test and a subsequent post-hoc test carried out on the ranked data
demonstrated that IR performed significantly better than the Feasibility Pump
heuristic when solving 152 benchmark problems of Nonconvex Mixed-Integer
Nonlinear Problems. However, is also showed that the RECIPE heuristic was
significantly better than IR when solving the same benchmark problems.
Learning Metric Graphs for Neuron Segmentation In Electron Microscopy Images
Authors: Kyle Luther, H. Sebastian Seung
Date: 2019-01-31 00:00:00
Tasks: Electron Microscopy Image Segmentation, Image Segmentation, Instance Segmentation, Metric Learning, Semantic Segmentation

In the deep metric learning approach to image segmentation, a convolutional
net densely generates feature vectors at the pixels of an image. Pairs of
feature vectors are trained to be similar or different, depending on whether
the corresponding pixels belong to same or different ground truth segments. To
segment a new image, the feature vectors are computed and clustered. Both
empirically and theoretically, it is unclear whether or when deep metric
learning is superior to the more conventional approach of directly predicting
an affinity graph with a convolutional net. We compare the two approaches using
brain images from serial section electron microscopy images, which constitute
an especially challenging example of instance segmentation. We first show that
seed-based postprocessing of the feature vectors, as originally proposed,
produces inferior accuracy because it is difficult for the convolutional net to
predict feature vectors that remain uniform across large objects. Then we
consider postprocessing by thresholding a nearest neighbor graph followed by
connected components. In this case, segmentations from a "metric graph" turn
out to be competitive or even superior to segmentations from a directly
predicted affinity graph. To explain these findings theoretically, we invoke
the property that the metric function satisfies the triangle inequality. Then
we show with an example where this constraint suppresses noise, causing
connected components to more robustly segment a metric graph than an
unconstrained affinity graph.
The Second Conversational Intelligence Challenge (ConvAI2)
Authors: Emily Dinan, Varvara Logacheva, Valentin Malykh, Alexander Miller, Kurt Shuster, Jack Urbanek, Douwe Kiela, Arthur Szlam, Iulian Serban, Ryan Lowe, Shrimai Prabhumoye, Alan W. black, Alexander Rudnicky, Jason Williams, Joelle Pineau, Mikhail Burtsev, Jason Weston
Date: 2019-01-31 00:00:00

We describe the setting and results of the ConvAI2 NeurIPS competition that
aims to further the state-of-the-art in open-domain chatbots. Some key
takeaways from the competition are: (i) pretrained Transformer variants are
currently the best performing models on this task, (ii) but to improve
performance on multi-turn conversations with humans, future systems must go
beyond single word metrics like perplexity to measure the performance across
sequences of utterances (conversations) -- in terms of repetition, consistency
and balance of dialogue acts (e.g. how many questions asked vs. answered).
Ensembling methods for countrywide short term forecasting of gas demand
Authors: Emanuele Fabbiani, Andrea Marziali, Giuseppe De Nicolao
Date: 2019-01-31 00:00:00
Tasks: Demand Forecasting, feature selection, Gaussian Processes, regression

Gas demand is made of three components: Residential, Industrial, and Thermoelectric Gas Demand. Herein, the one-day-ahead prediction of each component is studied, using Italian data as a case study. Statistical properties and relationships with temperature are discussed, as a preliminary step for an effective feature selection. Nine "base forecasters" are implemented and compared: Ridge Regression, Gaussian Processes, Nearest Neighbours, Artificial Neural Networks, Torus Model, LASSO, Elastic Net, Random Forest, and Support Vector Regression (SVR). Based on them, four ensemble predictors are crafted: simple average, weighted average, subset average, and SVR aggregation. We found that ensemble predictors perform consistently better than base ones. Moreover, our models outperformed Transmission System Operator (TSO) predictions in a two-year out-of-sample validation. Such results suggest that combining predictors may lead to significant performance improvements in gas demand forecasting.
Learning Triggers for Heterogeneous Treatment Effects
Authors: Christopher Tran, Elena Zheleva
Date: 2019-01-31 00:00:00
Tasks: Heterogeneous Treatment Effect Estimation, Recommendation Systems

The causal effect of a treatment can vary from person to person based on their individual characteristics and predispositions. Mining for patterns of individual-level effect differences, a problem known as heterogeneous treatment effect estimation, has many important applications, from precision medicine to recommender systems. In this paper we define and study a variant of this problem in which an individual-level threshold in treatment needs to be reached, in order to trigger an effect. One of the main contributions of our work is that we do not only estimate heterogeneous treatment effects with fixed treatments but can also prescribe individualized treatments. We propose a tree-based learning method to find the heterogeneity in the treatment effects. Our experimental results on multiple datasets show that our approach can learn the triggers better than existing approaches.
Minimax Testing of Identity to a Reference Ergodic Markov Chain
Authors: Geoffrey Wolfer, Aryeh Kontorovich
Date: 2019-01-31 00:00:00

We exhibit an efficient procedure for testing, based on a single long state sequence, whether an unknown Markov chain is identical to or $\varepsilon$-far from a given reference chain. We obtain nearly matching (up to logarithmic factors) upper and lower sample complexity bounds for our notion of distance, which is based on total variation. Perhaps surprisingly, we discover that the sample complexity depends solely on the properties of the known reference chain and does not involve the unknown chain at all, which is not even assumed to be ergodic.
Optimal mini-batch and step sizes for SAGA
Authors: Nidham Gazagnadou, Robert M. Gower, Joseph Salmon
Date: 2019-01-31 00:00:00

Recently it has been shown that the step sizes of a family of variance reduced gradient methods called the JacSketch methods depend on the expected smoothness constant. In particular, if this expected smoothness constant could be calculated a priori, then one could safely set much larger step sizes which would result in a much faster convergence rate. We fill in this gap, and provide simple closed form expressions for the expected smoothness constant and careful numerical experiments verifying these bounds. Using these bounds, and since the SAGA algorithm is part of this JacSketch family, we suggest a new standard practice for setting the step sizes and mini-batch size for SAGA that are competitive with a numerical grid search. Furthermore, we can now show that the total complexity of the SAGA algorithm decreases linearly in the mini-batch size up to a pre-defined value: the optimal mini-batch size. This is a rare result in the stochastic variance reduced literature, only previously shown for the Katyusha algorithm. Finally we conjecture that this is the case for many other stochastic variance reduced methods and that our bounds and analysis of the expected smoothness constant is key to extending these results.
Bayesian active learning for optimization and uncertainty quantification in protein docking
Authors: Yue Cao, Yang shen
Date: 2019-01-31 00:00:00
Tasks: Active Learning, Binary Classification, Uncertainty Quantification

Motivation: Ab initio protein docking represents a major challenge for
optimizing a noisy and costly "black box"-like function in a high-dimensional
space. Despite progress in this field, there is no docking method available for
rigorous uncertainty quantification (UQ) of its solution quality (e.g.
interface RMSD or iRMSD).
  Results: We introduce a novel algorithm, Bayesian Active Learning (BAL), for
optimization and UQ of such black-box functions and flexible protein docking.
BAL directly models the posterior distribution of the global optimum (or native
structures for protein docking) with active sampling and posterior estimation
iteratively feeding each other. Furthermore, we use complex normal modes to
represent a homogeneous Euclidean conformation space suitable for
high-dimension optimization and construct funnel-like energy models for
encounter complexes. Over a protein docking benchmark set and a CAPRI set
including homology docking, we establish that BAL significantly improve against
both starting points by rigid docking and refinements by particle swarm
optimization, providing for one third targets a top-3 near-native prediction.
BAL also generates tight confidence intervals with half range around 25% of
iRMSD and confidence level at 85%. Its estimated probability of a prediction
being native or not achieves binary classification AUROC at 0.93 and AUPRC over
0.60 (compared to 0.14 by chance); and also found to help ranking predictions.
To the best of our knowledge, this study represents the first uncertainty
quantification solution for protein docking, with theoretical rigor and
comprehensive assessment.
  Source codes are available at https://github.com/Shen-Lab/BAL.
Probabilistic Discriminative Learning with Layered Graphical Models
Authors: Yuesong Shen, Tao Wu, Csaba Domokos, Daniel Cremers
Date: 2019-01-31 00:00:00
Tasks: General Classification, image-classification, Image Classification, Variational Inference

Probabilistic graphical models are traditionally known for their successes in
generative modeling. In this work, we advocate layered graphical models (LGMs)
for probabilistic discriminative learning. To this end, we design LGMs in close
analogy to neural networks (NNs), that is, they have deep hierarchical
structures and convolutional or local connections between layers. Equipped with
tensorized truncated variational inference, our LGMs can be efficiently trained
via backpropagation on mainstream deep learning frameworks such as PyTorch. To
deal with continuous valued inputs, we use a simple yet effective soft-clamping
strategy for efficient inference. Through extensive experiments on image
classification over MNIST and FashionMNIST datasets, we demonstrate that LGMs
are capable of achieving competitive results comparable to NNs of similar
architectures, while preserving transparent probabilistic modeling.
Gaussian Conditional Random Fields for Classification
Authors: Andrija Petrović, Mladen Nikolić, Miloš Jovanović, Boris Delibašić
Date: 2019-01-31 00:00:00
Tasks: Binary Classification, Classification, General Classification

Gaussian conditional random fields (GCRF) are a well-known used structured
model for continuous outputs that uses multiple unstructured predictors to form
its features and at the same time exploits dependence structure among outputs,
which is provided by a similarity measure. In this paper, a Gaussian
conditional random fields model for structured binary classification (GCRFBC)
is proposed. The model is applicable to classification problems with undirected
graphs, intractable for standard classification CRFs. The model representation
of GCRFBC is extended by latent variables which yield some appealing
properties. Thanks to the GCRF latent structure, the model becomes tractable,
efficient and open to improvements previously applied to GCRF regression
models. In addition, the model allows for reduction of noise, that might appear
if structures were defined directly between discrete outputs. Additionally, two
different forms of the algorithm are presented: GCRFBCb (GCRGBC - Bayesian) and
GCRFBCnb (GCRFBC - non Bayesian). The extended method of local variational
approximation of sigmoid function is used for solving empirical Bayes in
Bayesian GCRFBCb variant, whereas MAP value of latent variables is the basis
for learning and inference in the GCRFBCnb variant. The inference in GCRFBCb is
solved by Newton-Cotes formulas for one-dimensional integration. Both models
are evaluated on synthetic data and real-world data. It was shown that both
models achieve better prediction performance than unstructured predictors.
Furthermore, computational and memory complexity is evaluated. Advantages and
disadvantages of the proposed GCRFBCb and GCRFBCnb are discussed in detail.
Your Gameplay Says It All: Modelling Motivation in Tom Clancy's The Division
Authors: David Melhart, Ahmad Azadvar, Alessandro Canossa, Antonios Liapis, Georgios N. Yannakakis
Date: 2019-01-31 00:00:00
Tasks: All

Is it possible to predict the motivation of players just by observing their gameplay data? Even if so, how should we measure motivation in the first place? To address the above questions, on the one end, we collect a large dataset of gameplay data from players of the popular game Tom Clancy's The Division. On the other end, we ask them to report their levels of competence, autonomy, relatedness and presence using the Ubisoft Perceived Experience Questionnaire. After processing the survey responses in an ordinal fashion we employ preference learning methods based on support vector machines to infer the mapping between gameplay and the reported four motivation factors. Our key findings suggest that gameplay features are strong predictors of player motivation as the best obtained models reach accuracies of near certainty, from 92% up to 94% on unseen players.
BLOCK: Bilinear Superdiagonal Fusion for Visual Question Answering and Visual Relationship Detection
Authors: Hedi Ben-Younes, Rémi Cadene, Nicolas Thome, Matthieu Cord
Date: 2019-01-31 00:00:00
Tasks: Question Answering, Relationship Detection, Representation Learning, Tensor Decomposition, Visual Question Answering, Visual Question Answering (VQA), Visual Relationship Detection

Multimodal representation learning is gaining more and more interest within
the deep learning community. While bilinear models provide an interesting
framework to find subtle combination of modalities, their number of parameters
grows quadratically with the input dimensions, making their practical
implementation within classical deep learning pipelines challenging. In this
paper, we introduce BLOCK, a new multimodal fusion based on the
block-superdiagonal tensor decomposition. It leverages the notion of block-term
ranks, which generalizes both concepts of rank and mode ranks for tensors,
already used for multimodal fusion. It allows to define new ways for optimizing
the tradeoff between the expressiveness and complexity of the fusion model, and
is able to represent very fine interactions between modalities while
maintaining powerful mono-modal representations. We demonstrate the practical
interest of our fusion model by using BLOCK for two challenging tasks: Visual
Question Answering (VQA) and Visual Relationship Detection (VRD), where we
design end-to-end learnable architectures for representing relevant
interactions between modalities. Through extensive experiments, we show that
BLOCK compares favorably with respect to state-of-the-art multimodal fusion
models for both VQA and VRD tasks. Our code is available at
https://github.com/Cadene/block.bootstrap.pytorch.
Compressed Diffusion
Authors: Scott Gigante, Jay S. Stanley III, Ngan Vu, David van Dijk, Kevin Moon, Guy Wolf, Smita Krishnaswamy
Date: 2019-01-31 00:00:00

Diffusion maps are a commonly used kernel-based method for manifold learning, which can reveal intrinsic structures in data and embed them in low dimensions. However, as with most kernel methods, its implementation requires a heavy computational load, reaching up to cubic complexity in the number of data points. This limits its usability in modern data analysis. Here, we present a new approach to computing the diffusion geometry, and related embeddings, from a compressed diffusion process between data regions rather than data points. Our construction is based on an adaptation of the previously proposed measure-based Gaussian correlation (MGC) kernel that robustly captures the local geometry around data points. We use this MGC kernel to efficiently compress diffusion relations from pointwise to data region resolution. Finally, a spectral embedding of the data regions provides coordinates that are used to interpolate and approximate the pointwise diffusion map embedding of data. We analyze theoretical connections between our construction and the original diffusion geometry of diffusion maps, and demonstrate the utility of our method in analyzing big datasets, where it outperforms competing approaches.
Towards Machine-assisted Meta-Studies: The Hubble Constant
Authors: Tom Crossland, Pontus Stenetorp, Sebastian Riedel, Daisuke Kawata, Thomas D. Kitching, Rupert A. C. Croft
Date: 2019-01-31 00:00:00

We present an approach for automatic extraction of measured values from the astrophysical literature, using the Hubble constant for our pilot study. Our rules-based model -- a classical technique in natural language processing -- has successfully extracted 298 measurements of the Hubble constant, with uncertainties, from the 208,541 available arXiv astrophysics papers. We have also created an artificial neural network classifier to identify papers in arXiv which report novel measurements. From the analysis of our results we find that reporting measurements with uncertainties and the correct units is critical information when distinguishing novel measurements in free text. Our results correctly highlight the current tension for measurements of the Hubble constant and recover the $3.5\sigma$ discrepancy -- demonstrating that the tool presented in this paper is useful for meta-studies of astrophysical measurements from a large number of publications.
Network Parameter Learning Using Nonlinear Transforms, Local Representation Goals and Local Propagation Constraints
Authors: Dimche Kostadinov, Behrooz Razdehi, Slava Voloshynovskiy
Date: 2019-01-31 00:00:00
Tasks: parameter estimation

In this paper, we introduce a novel concept for learning of the parameters in
a neural network. Our idea is grounded on modeling a learning problem that
addresses a trade-off between (i) satisfying local objectives at each node and
(ii) achieving desired data propagation through the network under (iii) local
propagation constraints. We consider two types of nonlinear transforms which
describe the network representations. One of the nonlinear transforms serves as
activation function. The other one enables a locally adjusted, deviation
corrective components to be included in the update of the network weights in
order to enable attaining target specific representations at the last network
node. Our learning principle not only provides insight into the understanding
and the interpretation of the learning dynamics, but it offers theoretical
guarantees over decoupled and parallel parameter estimation strategy that
enables learning in synchronous and asynchronous mode. Numerical experiments
validate the potential of our approach on image recognition task. The
preliminary results show advantages in comparison to the state-of-the-art
methods, w.r.t. the learning time and the network size while having competitive
recognition accuracy.