pwc-archive/papers-with-abstracts huggingface dataset:
Learning Representations from Dendrograms
Authors: Morteza Haghir Chehreghani, Mostafa Haghir Chehreghani
Date: 2018-12-21 00:00:00
Tasks: Clustering, Model Selection, Representation Learning

We propose unsupervised representation learning and feature extraction from dendrograms. The commonly used Minimax distance measures correspond to building a dendrogram with single linkage criterion, with defining specific forms of a level function and a distance function over that. Therefore, we extend this method to arbitrary dendrograms. We develop a generalized framework wherein different distance measures and representations can be inferred from different types of dendrograms, level functions and distance functions. Via an appropriate embedding, we compute a vector-based representation of the inferred distances, in order to enable many numerical machine learning algorithms to employ such distances. Then, to address the model selection problem, we study the aggregation of different dendrogram-based distances respectively in solution space and in representation space in the spirit of deep representations. In the first approach, for example for the clustering problem, we build a graph with positive and negative edge weights according to the consistency of the clustering labels of different objects among different solutions, in the context of ensemble methods. Then, we use an efficient variant of correlation clustering to produce the final clusters. In the second approach, we investigate the combination of different distances and features sequentially in the spirit of multi-layered architectures to obtain the final features. Finally, we demonstrate the effectiveness of our approach via several numerical studies.
Learning Compositional Representations for Few-Shot Recognition
Authors: Pavel Tokmakov, Yu-Xiong Wang, Martial Hebert
Date: 2018-12-21 00:00:00
Proceeding: ICCV 2019 10
Tasks: Attribute, Deep Learning, Few-Shot Learning

One of the key limitations of modern deep learning approaches lies in the amount of data required to train them. Humans, by contrast, can learn to recognize novel categories from just a few examples. Instrumental to this rapid learning ability is the compositional structure of concept representations in the human brain --- something that deep learning models are lacking. In this work, we make a step towards bridging this gap between human and machine learning by introducing a simple regularization technique that allows the learned representation to be decomposable into parts. Our method uses category-level attribute annotations to disentangle the feature space of a network into subspaces corresponding to the attributes. These attributes can be either purely visual, like object parts, or more abstract, like openness and symmetry. We demonstrate the value of compositional representations on three datasets: CUB-200-2011, SUN397, and ImageNet, and show that they require fewer examples to learn classifiers for novel categories.
Solution Dominance over Constraint Satisfaction Problems
Authors: Tias Guns, Peter J. Stuckey, Guido Tack
Date: 2018-12-21 00:00:00

Constraint Satisfaction Problems (CSPs) typically have many solutions that
satisfy all constraints. Often though, some solutions are preferred over
others, that is, some solutions dominate other solutions. We present solution
dominance as a formal framework to reason about such settings. We define
Constraint Dominance Problems (CDPs) as CSPs with a dominance relation, that
is, a preorder over the solutions of the CSP. This framework captures many
well-known variants of constraint satisfaction, including optimization,
multi-objective optimization, Max-CSP, minimal models, minimum correction
subsets as well as optimization over CP-nets and arbitrary dominance relations.
We extend MiniZinc, a declarative language for modeling CSPs, to CDPs by
introducing dominance nogoods; these can be derived from dominance relations in
a principled way. A generic method for solving arbitrary CDPs incrementally
calls a CSP solver and is compatible with any existing solver that supports
MiniZinc. This encourages experimenting with different solution dominance
relations for a problem, as well as comparing different solvers without having
to modify their implementations.
Learning to Navigate the Web
Authors: Izzeddin Gur, Ulrich Rueckert, Aleksandra Faust, Dilek Hakkani-Tur
Date: 2018-12-21 00:00:00
Proceeding: ICLR 2019 5
Tasks: Deep Reinforcement Learning, Instruction Following, Meta-Learning, Navigate, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Learning in environments with large state and action spaces, and sparse
rewards, can hinder a Reinforcement Learning (RL) agent's learning through
trial-and-error. For instance, following natural language instructions on the
Web (such as booking a flight ticket) leads to RL settings where input
vocabulary and number of actionable elements on a page can grow very large.
Even though recent approaches improve the success rate on relatively simple
environments with the help of human demonstrations to guide the exploration,
they still fail in environments where the set of possible instructions can
reach millions. We approach the aforementioned problems from a different
perspective and propose guided RL approaches that can generate unbounded amount
of experience for an agent to learn from. Instead of learning from a
complicated instruction with a large vocabulary, we decompose it into multiple
sub-instructions and schedule a curriculum in which an agent is tasked with a
gradually increasing subset of these relatively easier sub-instructions. In
addition, when the expert demonstrations are not available, we propose a novel
meta-learning framework that generates new instruction following tasks and
trains the agent more effectively. We train DQN, deep reinforcement learning
agent, with Q-value function approximated with a novel QWeb neural network
architecture on these smaller, synthetic instructions. We evaluate the ability
of our agent to generalize to new instructions on World of Bits benchmark, on
forms with up to 100 elements, supporting 14 million possible instructions. The
QWeb agent outperforms the baseline without using any human demonstration
achieving 100% success rate on several difficult environments.
Sources of Complexity in Semantic Frame Parsing for Information Extraction
Authors: Gabriel Marzinotto, Frédéric Béchet, Géraldine Damnati, Alexis Nasr
Date: 2018-12-21 00:00:00
Tasks: Semantic Frame Parsing

This paper describes a Semantic Frame parsing System based on sequence
labeling methods, precisely BiLSTM models with highway connections, for
performing information extraction on a corpus of French encyclopedic history
texts annotated according to the Berkeley FrameNet formalism. The approach
proposed in this study relies on an integrated sequence labeling model which
jointly optimizes frame identification and semantic role segmentation and
identification. The purpose of this study is to analyze the task complexity, to
highlight the factors that make Semantic Frame parsing a difficult task and to
provide detailed evaluations of the performance on different types of frames
and sentences.
Quicker ADC : Unlocking the hidden potential of Product Quantization with SIMD
Authors: Fabien André, Anne-Marie Kermarrec, Nicolas Le Scouarnec
Date: 2018-12-21 00:00:00
Tasks: Quantization, Retrieval

Efficient Nearest Neighbor (NN) search in high-dimensional spaces is a foundation of many multimedia retrieval systems. A common approach is to rely on Product Quantization, which allows the storage of large vector databases in memory and efficient distance computations. Yet, implementations of nearest neighbor search with Product Quantization have their performance limited by the many memory accesses they perform. Following this observation, Andr\'e et al. proposed Quick ADC with up to $6\times$ faster implementations of $m\times{}4$ product quantizers (PQ) leveraging specific SIMD instructions. Quicker ADC is a generalization of Quick ADC not limited to $m\times{}4$ codes and supporting AVX-512, the latest revision of SIMD instruction set. In doing so, Quicker ADC faces the challenge of using efficiently 5,6 and 7-bit shuffles that do not align to computer bytes or words. To this end, we introduce (i) irregular product quantizers combining sub-quantizers of different granularity and (ii) split tables allowing lookup tables larger than registers. We evaluate Quicker ADC with multiple indexes including Inverted Multi-Indexes and IVF HNSW and show that it outperforms the reference optimized implementations (i.e., FAISS and polysemous codes) for numerous configurations. Finally, we release an open-source fork of FAISS enhanced with Quicker ADC at http://github.com/nlescoua/faiss-quickeradc.
Ecological Data Analysis Based on Machine Learning Algorithms
Authors: Md. Siraj-Ud-Doula, Md. Ashad Alam
Date: 2018-12-21 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification

Classification is an important supervised machine learning method, which is
necessary and challenging issue for ecological research. It offers a way to
classify a dataset into subsets that share common patterns. Notably, there are
many classification algorithms to choose from, each making certain assumptions
about the data and about how classification should be formed. In this paper, we
applied eight machine learning classification algorithms such as Decision
Trees, Random Forest, Artificial Neural Network, Support Vector Machine, Linear
Discriminant Analysis, k-nearest neighbors, Logistic Regression and Naive Bayes
on ecological data. The goal of this study is to compare different machine
learning classification algorithms in ecological dataset. In this analysis we
have checked the accuracy test among the algorithms. In our study we conclude
that Linear Discriminant Analysis and k-nearest neighbors are the best methods
among all other methods
A Multiscale Image Denoising Algorithm Based On Dilated Residual Convolution Network
Authors: Chang Liu, Zhaowei Shang, Anyong Qin
Date: 2018-12-21 00:00:00
Tasks: Denoising, Image Denoising

Image denoising is a classical problem in low level computer vision.
Model-based optimization methods and deep learning approaches have been the two
main strategies for solving the problem. Model-based optimization methods are
flexible for handling different inverse problems but are usually
time-consuming. In contrast, deep learning methods have fast testing speed but
the performance of these CNNs is still inferior. To address this issue, here we
propose a novel deep residual learning model that combines the dilated residual
convolution and multi-scale convolution groups. Due to the complex patterns and
structures of inside an image, the multiscale convolution group is utilized to
learn those patterns and enlarge the receptive field. Specifically, the
residual connection and batch normalization are utilized to speed up the
training process and maintain the denoising performance. In order to decrease
the gridding artifacts, we integrate the hybrid dilated convolution design into
our model. To this end, this paper aims to train a lightweight and effective
denoiser based on multiscale convolution group. Experimental results have
demonstrated that the enhanced denoiser can not only achieve promising
denoising results, but also become a strong competitor in practical
application.
Cascaded Coarse-to-Fine Deep Kernel Networks for Efficient Satellite Image Change Detection
Authors: Hichem Sahbi
Date: 2018-12-21 00:00:00
Tasks: Change Detection

Deep networks are nowadays becoming popular in many computer vision and
pattern recognition tasks. Among these networks, deep kernels are particularly
interesting and effective, however, their computational complexity is a major
issue especially on cheap hardware resources. In this paper, we address the
issue of efficient computation in deep kernel networks. We propose a novel
framework that reduces dramatically the complexity of evaluating these deep
kernels. Our method is based on a coarse-to-fine cascade of networks designed
for efficient computation; early stages of the cascade are cheap and reject
many patterns efficiently while deep stages are more expensive and accurate.
The design principle of these reduced complexity networks is based on a variant
of the cross-entropy criterion that reduces the complexity of the networks in
the cascade while preserving all the positive responses of the original kernel
network. Experiments conducted - on the challenging and time demanding change
detection task, on very large satellite images - show that our proposed
coarse-to-fine approach is effective and highly efficient.
Introducing Neuromodulation in Deep Neural Networks to Learn Adaptive Behaviours
Authors: Nicolas Vecoven, Damien Ernst, Antoine Wehenkel, Guillaume Drion
Date: 2018-12-21 00:00:00
Tasks: Meta Reinforcement Learning, Reinforcement Learning

Animals excel at adapting their intentions, attention, and actions to the environment, making them remarkably efficient at interacting with a rich, unpredictable and ever-changing external world, a property that intelligent machines currently lack. Such an adaptation property relies heavily on cellular neuromodulation, the biological mechanism that dynamically controls intrinsic properties of neurons and their response to external stimuli in a context-dependent manner. In this paper, we take inspiration from cellular neuromodulation to construct a new deep neural network architecture that is specifically designed to learn adaptive behaviours. The network adaptation capabilities are tested on navigation benchmarks in a meta-reinforcement learning context and compared with state-of-the-art approaches. Results show that neuromodulation is capable of adapting an agent to different tasks and that neuromodulation-based approaches provide a promising way of improving adaptation of artificial systems.
Generative Models from the perspective of Continual Learning
Authors: Timothée Lesort, Hugo Caselles-Dupré, Michael Garcia-Ortiz, Andrei Stoian, David Filliat
Date: 2018-12-21 00:00:00
Proceeding: ICLR 2019 5
Tasks: Continual Learning, Image Generation

Which generative model is the most suitable for Continual Learning? This
paper aims at evaluating and comparing generative models on disjoint sequential
image generation tasks. We investigate how several models learn and forget,
considering various strategies: rehearsal, regularization, generative replay
and fine-tuning. We used two quantitative metrics to estimate the generation
quality and memory ability. We experiment with sequential tasks on three
commonly used benchmarks for Continual Learning (MNIST, Fashion MNIST and
CIFAR10). We found that among all models, the original GAN performs best and
among Continual Learning strategies, generative replay outperforms all other
methods. Even if we found satisfactory combinations on MNIST and Fashion MNIST,
training generative models sequentially on CIFAR10 is particularly instable,
and remains a challenge. Our code is available online
\footnote{\url{https://github.com/TLESORT/Generative\_Continual\_Learning}}.
Reasoning and Facts Explanation in Valuation Based Systems
Authors: S. T. Wierzchoń, M. A. Kłopotek, M. Michalewicz
Date: 2018-12-21 00:00:00

In the literature, the optimization problem to identify a set of composite
hypotheses H, which will yield the $k$ largest $P(H|S_e)$ where a composite
hypothesis is an instantiation of all the nodes in the network except the
evidence nodes \cite{KSy:93} is of significant interest. This problem is called
"finding the $k$ Most Plausible Explanation (MPE) of a given evidence $S_e$ in
a Bayesian belief network".
  The problem of finding $k$ most probable hypotheses is generally NP-hard
\cite{Cooper:90}. Therefore in the past various simplifications of the task by
restricting $k$ (to 1 or 2), restricting the structure (e.g. to singly
connected networks), or shifting the complexity to spatial domain have been
investigated.
  A genetic algorithm is proposed in this paper to overcome some of these
restrictions while stepping out from probabilistic domain onto the general
Valuation based System (VBS) framework is also proposed by generalizing the
genetic algorithm approach to the realm of Dempster-Shafer belief calculus.
3DSRnet: Video Super-resolution using 3D Convolutional Neural Networks
Authors: Soo Ye Kim, Jeongyeon Lim, Taeyoung Na, Munchurl Kim
Date: 2018-12-21 00:00:00
Tasks: Super-Resolution, Video Super-Resolution

In video super-resolution, the spatio-temporal coherence between, and among the frames must be exploited appropriately for accurate prediction of the high resolution frames. Although 2D convolutional neural networks (CNNs) are powerful in modelling images, 3D-CNNs are more suitable for spatio-temporal feature extraction as they can preserve temporal information. To this end, we propose an effective 3D-CNN for video super-resolution, called the 3DSRnet that does not require motion alignment as preprocessing. Our 3DSRnet maintains the temporal depth of spatio-temporal feature maps to maximally capture the temporally nonlinear characteristics between low and high resolution frames, and adopts residual learning in conjunction with the sub-pixel outputs. It outperforms the most state-of-the-art method with average 0.45 and 0.36 dB higher in PSNR for scales 3 and 4, respectively, in the Vidset4 benchmark. Our 3DSRnet first deals with the performance drop due to scene change, which is important in practice but has not been previously considered.
Dreaming neural networks: rigorous results
Authors: Elena Agliari, Francesco Alemanno, Adriano Barra, Alberto Fachechi
Date: 2018-12-21 00:00:00
Tasks: Retrieval

Recently a daily routine for associative neural networks has been proposed:
the network Hebbian-learns during the awake state (thus behaving as a standard
Hopfield model), then, during its sleep state, optimizing information storage,
it consolidates pure patterns and removes spurious ones: this forces the
synaptic matrix to collapse to the projector one (ultimately approaching the
Kanter-Sompolinksy model). This procedure keeps the learning Hebbian-based (a
biological must) but, by taking advantage of a (properly stylized) sleep phase,
still reaches the maximal critical capacity (for symmetric interactions). So
far this emerging picture (as well as the bulk of papers on unlearning
techniques) was supported solely by mathematically-challenging routes, e.g.
mainly replica-trick analysis and numerical simulations: here we rely
extensively on Guerra's interpolation techniques developed for neural networks
and, in particular, we extend the generalized stochastic stability approach to
the case. Confining our description within the replica symmetric approximation
(where the previous ones lie), the picture painted regarding this
generalization (and the previously existing variations on theme) is here
entirely confirmed. Further, still relying on Guerra's schemes, we develop a
systematic fluctuation analysis to check where ergodicity is broken (an
analysis entirely absent in previous investigations). We find that, as long as
the network is awake, ergodicity is bounded by the Amit-Gutfreund-Sompolinsky
critical line (as it should), but, as the network sleeps, sleeping destroys
spin glass states by extending both the retrieval as well as the ergodic
region: after an entire sleeping session the solely surviving regions are
retrieval and ergodic ones and this allows the network to achieve the perfect
retrieval regime (the number of storable patterns equals the number of neurons
in the network).
An Integrated Transfer Learning and Multitask Learning Approach for Pharmacokinetic Parameter Prediction
Authors: Zhuyifan Ye, Yilong Yang, Xiaoshan Li, Dongsheng Cao, Defang Ouyang
Date: 2018-12-21 00:00:00
Tasks: Drug Discovery, Parameter Prediction, Transfer Learning

Background: Pharmacokinetic evaluation is one of the key processes in drug
discovery and development. However, current absorption, distribution,
metabolism, excretion prediction models still have limited accuracy. Aim: This
study aims to construct an integrated transfer learning and multitask learning
approach for developing quantitative structure-activity relationship models to
predict four human pharmacokinetic parameters. Methods: A pharmacokinetic
dataset included 1104 U.S. FDA approved small molecule drugs. The dataset
included four human pharmacokinetic parameter subsets (oral bioavailability,
plasma protein binding rate, apparent volume of distribution at steady-state
and elimination half-life). The pre-trained model was trained on over 30
million bioactivity data. An integrated transfer learning and multitask
learning approach was established to enhance the model generalization. Results:
The pharmacokinetic dataset was split into three parts (60:20:20) for training,
validation and test by the improved Maximum Dissimilarity algorithm with the
representative initial set selection algorithm and the weighted distance
function. The multitask learning techniques enhanced the model predictive
ability. The integrated transfer learning and multitask learning model
demonstrated the best accuracies, because deep neural networks have the general
feature extraction ability, transfer learning and multitask learning improved
the model generalization. Conclusions: The integrated transfer learning and
multitask learning approach with the improved dataset splitting algorithm was
firstly introduced to predict the pharmacokinetic parameters. This method can
be further employed in drug discovery and development.
Marvels and Pitfalls of the Langevin Algorithm in Noisy High-dimensional Inference
Authors: Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, Lenka Zdeborová
Date: 2018-12-21 00:00:00

Gradient-descent-based algorithms and their stochastic versions have widespread applications in machine learning and statistical inference. In this work we perform an analytic study of the performances of one of them, the Langevin algorithm, in the context of noisy high-dimensional inference. We employ the Langevin algorithm to sample the posterior probability measure for the spiked matrix-tensor model. The typical behaviour of this algorithm is described by a system of integro-differential equations that we call the Langevin state evolution, whose solution is compared with the one of the state evolution of approximate message passing (AMP). Our results show that, remarkably, the algorithmic threshold of the Langevin algorithm is sub-optimal with respect to the one given by AMP. We conjecture this phenomenon to be due to the residual glassiness present in that region of parameters. Finally we show how a landscape-annealing protocol, that uses the Langevin algorithm but violate the Bayes-optimality condition, can approach the performance of AMP.
GaussianProcesses.jl: A Nonparametric Bayes package for the Julia Language
Authors: Jamie Fairbrother, Christopher Nemeth, Maxime Rischard, Johanni Brea, Thomas Pinder
Date: 2018-12-21 00:00:00
Tasks: Binary Classification, Gaussian Processes

Gaussian processes are a class of flexible nonparametric Bayesian tools that are widely used across the sciences, and in industry, to model complex data sources. Key to applying Gaussian process models is the availability of well-developed open source software, which is available in many programming languages. In this paper, we present a tutorial of the GaussianProcesses.jl package that has been developed for the Julia programming language. GaussianProcesses.jl utilises the inherent computational benefits of the Julia language, including multiple dispatch and just-in-time compilation, to produce a fast, flexible and user-friendly Gaussian processes package. The package provides many mean and kernel functions with supporting inference tools to fit exact Gaussian process models, as well as a range of alternative likelihood functions to handle non-Gaussian data (e.g. binary classification models) and sparse approximations for scalable Gaussian processes. The package makes efficient use of existing Julia packages to provide users with a range of optimization and plotting tools.