pwc-archive/papers-with-abstracts huggingface dataset:
Learning Product Codebooks using Vector Quantized Autoencoders for Image Retrieval
Authors: Hanwei Wu, Markus Flierl
Date: 2018-07-12 00:00:00
Tasks: Image Retrieval, Quantization, Representation Learning, Retrieval

Vector-Quantized Variational Autoencoders (VQ-VAE)[1] provide an unsupervised
model for learning discrete representations by combining vector quantization
and autoencoders. In this paper, we study the use of VQ-VAE for representation
learning for downstream tasks, such as image retrieval. We first describe the
VQ-VAE in the context of an information-theoretic framework. We show that the
regularization term on the learned representation is determined by the size of
the embedded codebook before the training and it affects the generalization
ability of the model. As a result, we introduce a hyperparameter to balance the
strength of the vector quantizer and the reconstruction error. By tuning the
hyperparameter, the embedded bottleneck quantizer is used as a regularizer that
forces the output of the encoder to share a constrained coding space such that
learned latent features preserve the similarity relations of the data space. In
addition, we provide a search range for finding the best hyperparameter.
Finally, we incorporate the product quantization into the bottleneck stage of
VQ-VAE and propose an end-to-end unsupervised learning model for the image
retrieval task. The product quantizer has the advantage of generating
large-size codebooks. Fast retrieval can be achieved by using the lookup tables
that store the distance between any pair of sub-codewords. State-of-the-art
retrieval results are achieved by the learned codebooks.
Scikit-Multiflow: A Multi-output Streaming Framework
Authors: Jacob Montiel, Jesse Read, Albert Bifet, Talel Abdessalem
Date: 2018-07-12 00:00:00

Scikit-multiflow is a multi-output/multi-label and stream data mining
framework for the Python programming language. Conceived to serve as a platform
to encourage democratization of stream learning research, it provides multiple
state of the art methods for stream learning, stream generators and evaluators.
scikit-multiflow builds upon popular open source frameworks including
scikit-learn, MOA and MEKA. Development follows the FOSS principles and quality
is enforced by complying with PEP8 guidelines and using continuous integration
and automatic testing. The source code is publicly available at
https://github.com/scikit-multiflow/scikit-multiflow.
Deep semi-supervised segmentation with weight-averaged consistency targets
Authors: Christian S. Perone, Julien Cohen-Adad
Date: 2018-07-12 00:00:00
Tasks: Data Augmentation, General Classification, Segmentation, Small Data Image Classification

Recently proposed techniques for semi-supervised learning such as Temporal
Ensembling and Mean Teacher have achieved state-of-the-art results in many
important classification benchmarks. In this work, we expand the Mean Teacher
approach to segmentation tasks and show that it can bring important
improvements in a realistic small data regime using a publicly available
multi-center dataset from the Magnetic Resonance Imaging (MRI) domain. We also
devise a method to solve the problems that arise when using traditional data
augmentation strategies for segmentation tasks on our new training scheme.
Algorithms that Remember: Model Inversion Attacks and Data Protection Law
Authors: Michael Veale, Reuben Binns, Lilian Edwards
Date: 2018-07-12 00:00:00

Many individuals are concerned about the governance of machine learning
systems and the prevention of algorithmic harms. The EU's recent General Data
Protection Regulation (GDPR) has been seen as a core tool for achieving better
governance of this area. While the GDPR does apply to the use of models in some
limited situations, most of its provisions relate to the governance of personal
data, while models have traditionally been seen as intellectual property. We
present recent work from the information security literature around `model
inversion' and `membership inference' attacks, which indicate that the process
of turning training data into machine learned systems is not one-way, and
demonstrate how this could lead some models to be legally classified as
personal data. Taking this as a probing experiment, we explore the different
rights and obligations this would trigger and their utility, and posit future
directions for algorithmic governance and regulation.
Automatically Composing Representation Transformations as a Means for Generalization
Authors: Michael B. Chang, Abhishek Gupta, Sergey Levine, Thomas L. Griffiths
Date: 2018-07-12 00:00:00
Proceeding: ICLR 2019 5
Tasks: Decision Making

A generally intelligent learner should generalize to more complex tasks than it has previously encountered, but the two common paradigms in machine learning -- either training a separate learner per task or training a single learner for all tasks -- both have difficulty with such generalization because they do not leverage the compositional structure of the task distribution. This paper introduces the compositional problem graph as a broadly applicable formalism to relate tasks of different complexity in terms of problems with shared subproblems. We propose the compositional generalization problem for measuring how readily old knowledge can be reused and hence built upon. As a first step for tackling compositional generalization, we introduce the compositional recursive learner, a domain-general framework for learning algorithmic procedures for composing representation transformations, producing a learner that reasons about what computation to execute by making analogies to previously seen problems. We show on a symbolic and a high-dimensional domain that our compositional approach can generalize to more complex problems than the learner has previously encountered, whereas baselines that are not explicitly compositional do not.
Rule Induction Partitioning Estimator
Authors: Vincent Margot, Jean-Patrick Baudry, Frederic Guilloux, Olivier Wintenberger
Date: 2018-07-12 00:00:00

RIPE is a novel deterministic and easily understandable prediction algorithm
developed for continuous and discrete ordered data. It infers a model, from a
sample, to predict and to explain a real variable $Y$ given an input variable
$X \in \mathcal X$ (features). The algorithm extracts a sparse set of
hyperrectangles $\mathbf r \subset \mathcal X$, which can be thought of as
rules of the form If-Then. This set is then turned into a partition of the
features space $\mathcal X$ of which each cell is explained as a list of rules
with satisfied their If conditions. The process of RIPE is illustrated on
simulated datasets and its efficiency compared with that of other usual
algorithms.
The Incremental Proximal Method: A Probabilistic Perspective
Authors: Ömer Deniz Akyildiz, Victor Elvira, Joaquin Miguez
Date: 2018-07-12 00:00:00
Tasks: regression

In this work, we highlight a connection between the incremental proximal
method and stochastic filters. We begin by showing that the proximal operators
coincide, and hence can be realized with, Bayes updates. We give the explicit
form of the updates for the linear regression problem and show that there is a
one-to-one correspondence between the proximal operator of the least-squares
regression and the Bayes update when the prior and the likelihood are Gaussian.
We then carry out this observation to a general sequential setting: We consider
the incremental proximal method, which is an algorithm for large-scale
optimization, and show that, for a linear-quadratic cost function, it can
naturally be realized by the Kalman filter. We then discuss the implications of
this idea for nonlinear optimization problems where proximal operators are in
general not realizable. In such settings, we argue that the extended Kalman
filter can provide a systematic way for the derivation of practical procedures.
Assessing the Scalability of Biologically-Motivated Deep Learning Algorithms and Architectures
Authors: Sergey Bartunov, Adam Santoro, Blake A. Richards, Luke Marris, Geoffrey E. Hinton, Timothy Lillicrap
Date: 2018-07-12 00:00:00
Proceeding: NeurIPS 2018 12
Tasks: Deep Learning

The backpropagation of error algorithm (BP) is impossible to implement in a
real brain. The recent success of deep networks in machine learning and AI,
however, has inspired proposals for understanding how the brain might learn
across multiple layers, and hence how it might approximate BP. As of yet, none
of these proposals have been rigorously evaluated on tasks where BP-guided deep
learning has proved critical, or in architectures more structured than simple
fully-connected networks. Here we present results on scaling up biologically
motivated models of deep learning on datasets which need deep networks with
appropriate architectures to achieve good performance. We present results on
the MNIST, CIFAR-10, and ImageNet datasets and explore variants of
target-propagation (TP) and feedback alignment (FA) algorithms, and explore
performance in both fully- and locally-connected architectures. We also
introduce weight-transport-free variants of difference target propagation (DTP)
modified to remove backpropagation from the penultimate layer. Many of these
algorithms perform well for MNIST, but for CIFAR and ImageNet we find that TP
and FA variants perform significantly worse than BP, especially for networks
composed of locally connected units, opening questions about whether new
architectures and algorithms are required to scale these approaches. Our
results and implementation details help establish baselines for biologically
motivated deep learning schemes going forward.
Deep Learning for Imbalance Data Classification using Class Expert Generative Adversarial Network
Authors: Fanny, Tjeng Wawan Cenggoro
Date: 2018-07-12 00:00:00
Tasks: Classification, General Classification, Generative Adversarial Network

Without any specific way for imbalance data classification, artificial
intelligence algorithm cannot recognize data from minority classes easily. In
general, modifying the existing algorithm by assuming that the training data is
imbalanced, is the only way to handle imbalance data. However, for a normal
data handling, this way mostly produces a deficient result. In this research,
we propose a class expert generative adversarial network (CE-GAN) as the
solution for imbalance data classification. CE-GAN is a modification in deep
learning algorithm architecture that does not have an assumption that the
training data is imbalance data. Moreover, CE-GAN is designed to identify more
detail about the character of each class before classification step. CE-GAN has
been proved in this research to give a good performance for imbalance data
classification.
Decentralized Clustering on Compressed Data without Prior Knowledge of the Number of Clusters
Authors: Elsa Dupraz, Dominique Pastor, François-Xavier Socheleau
Date: 2018-07-12 00:00:00
Tasks: Clustering

In sensor networks, it is not always practical to set up a fusion center.
Therefore, there is need for fully decentralized clustering algorithms.
Decentralized clustering algorithms should minimize the amount of data
exchanged between sensors in order to reduce sensor energy consumption. In this
respect, we propose one centralized and one decentralized clustering algorithm
that work on compressed data without prior knowledge of the number of clusters.
In the standard K-means clustering algorithm, the number of clusters is
estimated by repeating the algorithm several times, which dramatically
increases the amount of exchanged data, while our algorithm can estimate this
number in one run.
  The proposed clustering algorithms derive from a theoretical framework
establishing that, under asymptotic conditions, the cluster centroids are the
only fixed-point of a cost function we introduce. This cost function depends on
a weight function which we choose as the p-value of a Wald hypothesis test.
This p-value measures the plausibility that a given measurement vector belongs
to a given cluster. Experimental results show that our two algorithms are
competitive in terms of clustering performance with respect to K-means and
DB-Scan, while lowering by a factor at least $2$ the amount of data exchanged
between sensors.
Robustness Analysis of Pedestrian Detectors for Surveillance
Authors: Yuming Fang, Guanqun Ding, Yuan Yuan, Weisi Lin, Haiwen Liu
Date: 2018-07-12 00:00:00
Tasks: Pedestrian Detection

To obtain effective pedestrian detection results in surveillance video, there
have been many methods proposed to handle the problems from severe occlusion,
pose variation, clutter background, \emph{etc}. Besides detection accuracy, a
robust surveillance video system should be stable to video quality degradation
by network transmission, environment variation, etc. In this study, we conduct
the research on the robustness of pedestrian detection algorithms to video
quality degradation. The main contribution of this work includes the following
three aspects. First, a large-scale Distorted Surveillance Video Data Set
(DSurVD) is constructed from high-quality video sequences and their
corresponding distorted versions. Second, we design a method to evaluate
detection stability and a robustness measure called Robustness Quadrangle,
which can be adopted to visualize detection accuracy of pedestrian detection
algorithms on high-quality video sequences and stability with video quality
degradation. Third, the robustness of seven existing pedestrian detection
algorithms is evaluated by the built DSurVD. Experimental results show that the
robustness can be further improved for existing pedestrian detection
algorithms. Additionally, we provide much in-depth discussion on how different
distortion types influence the performance of pedestrian detection algorithms,
which is important to design effective pedestrian detection algorithms for
surveillance. The DSurVD data set can be download from BaiduYunDisk,
https://pan.baidu.com/s/1I9Kqj8rmubOYu7bkBfkUpA, Password: lqmc
Situation Calculus for Synthesis of Manufacturing Controllers
Authors: Giuseppe De Giacomo, Brian Logan, Paolo Felli, Fabio Patrizi, Sebastian Sardina
Date: 2018-07-12 00:00:00

Manufacturing is transitioning from a mass production model to a
manufacturing as a service model in which manufacturing facilities 'bid' to
produce products. To decide whether to bid for a complex, previously unseen
product, a manufacturing facility must be able to synthesize, 'on the fly', a
process plan controller that delegates abstract manufacturing tasks in the
supplied process recipe to the appropriate manufacturing resources, e.g., CNC
machines, robots etc. Previous work in applying AI behaviour composition to
synthesize process plan controllers has considered only finite state ad-hoc
representations. Here, we study the problem in the relational setting of the
Situation Calculus. By taking advantage of recent work on abstraction in the
Situation Calculus, process recipes and available resources are represented by
ConGolog programs over, respectively, an abstract and a concrete action theory.
This allows us to capture the problem in a formal, general framework, and show
decidability for the case of bounded action theories. We also provide
techniques for actually synthesizing the controller.
A Constrained Randomized Shortest-Paths Framework for Optimal Exploration
Authors: Bertrand Lebichot, Guillaume Guex, Ilkka Kivimäki, Marco Saerens
Date: 2018-07-12 00:00:00
Tasks: Reinforcement Learning

The present work extends the randomized shortest-paths framework (RSP),
interpolating between shortest-path and random-walk routing in a network, in
three directions. First, it shows how to deal with equality constraints on a
subset of transition probabilities and develops a generic algorithm for solving
this constrained RSP problem using Lagrangian duality. Second, it derives a
surprisingly simple iterative procedure to compute the optimal, randomized,
routing policy generalizing the previously developed "soft" Bellman-Ford
algorithm. The resulting algorithm allows balancing exploitation and
exploration in an optimal way by interpolating between a pure random behavior
and the deterministic, optimal, policy (least-cost paths) while satisfying the
constraints. Finally, the two algorithms are applied to Markov decision
problems by considering the process as a constrained RSP on a bipartite
state-action graph. In this context, the derived "soft" value iteration
algorithm appears to be closely related to dynamic policy programming as well
as Kullback-Leibler and path integral control, and similar to a recently
introduced reinforcement learning exploration strategy. This shows that this
strategy is optimal in the RSP sense - it minimizes expected path cost subject
to relative entropy constraint. Simulation results on illustrative examples
show that the model behaves as expected.
Video Saliency Detection by 3D Convolutional Neural Networks
Authors: Guanqun Ding, Yuming Fang
Date: 2018-07-12 00:00:00
Tasks: Object, object-detection, Object Detection, RGB Salient Object Detection, Saliency Detection, Saliency Prediction, Salient Object Detection, Video Saliency Detection, Video Saliency Prediction

Different from salient object detection methods for still images, a key
challenging for video saliency detection is how to extract and combine spatial
and temporal features. In this paper, we present a novel and effective approach
for salient object detection for video sequences based on 3D convolutional
neural networks. First, we design a 3D convolutional network (Conv3DNet) with
the input as three video frame to learn the spatiotemporal features for video
sequences. Then, we design a 3D deconvolutional network (Deconv3DNet) to
combine the spatiotemporal features to predict the final saliency map for video
sequences. Experimental results show that the proposed saliency detection model
performs better in video saliency prediction compared with the state-of-the-art
video saliency detection methods.
Training Neural Networks Using Features Replay
Authors: Zhouyuan Huo, Bin Gu, Heng Huang
Date: 2018-07-12 00:00:00
Proceeding: NeurIPS 2018 12

Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network. The backward locking prevents us from updating network layers in parallel and fully leveraging the computing resources. Recently, there are several works trying to decouple and parallelize the backpropagation algorithm. However, all of them suffer from severe accuracy loss or memory explosion when the neural network is deep. To address these challenging issues, we propose a novel parallel-objective formulation for the objective function of the neural network. After that, we introduce features replay algorithm and prove that it is guaranteed to converge to critical points for the non-convex problem under certain conditions. Finally, we apply our method to training deep convolutional neural networks, and the experimental results show that the proposed method achieves {faster} convergence, {lower} memory consumption, and {better} generalization error than compared methods.
Achieving Connectivity Between Wide Areas Through Self-Organising Robot Swarm Using Embodied Evolution
Authors: Erik Aaron Hansen, Stefano Nichele, Anis Yazidi, Hårek Haugerud, Asieh Abolpour Mofrad, Alex Alcocer
Date: 2018-07-12 00:00:00
Tasks: Artificial Life

Abruptions to the communication infrastructure happens occasionally, where
manual dedicated personnel will go out to fix the interruptions, restoring
communication abilities. However, sometimes this can be dangerous to the
personnel carrying out the task, which can be the case in war situations,
environmental disasters like earthquakes or toxic spills or in the occurrence
of fire. Therefore, human casualties can be minimised if autonomous robots are
deployed that can achieve the same outcome: to establish a communication link
between two previously distant but connected sites. In this paper we
investigate the deployment of mobile ad hoc robots which relay traffic between
them. In order to get the robots to locate themselves appropriately, we take
inspiration from self-organisation and emergence in artificial life, where a
common overall goal may be achieved if the correct local rules on the agents in
system are invoked. We integrate the aspect of connectivity between two sites
into the multirobot simulation platform known as JBotEvolver. The robot swarm
is composed of Thymio II robots. In addition, we compare three heuristics, of
which one uses neuroevolution (evolution of neural networks) to show how
self-organisation and embodied evolution can be used within the integration.
Our use of embodiment in robotic controllers shows promising results and
provide solid knowledge and guidelines for further investigations.
Fast yet Simple Natural-Gradient Descent for Variational Inference in Complex Models
Authors: Mohammad Emtiyaz Khan, Didrik Nielsen
Date: 2018-07-12 00:00:00
Tasks: Bayesian Inference, Variational Inference

Bayesian inference plays an important role in advancing machine learning, but
faces computational challenges when applied to complex models such as deep
neural networks. Variational inference circumvents these challenges by
formulating Bayesian inference as an optimization problem and solving it using
gradient-based optimization. In this paper, we argue in favor of
natural-gradient approaches which, unlike their gradient-based counterparts,
can improve convergence by exploiting the information geometry of the
solutions. We show how to derive fast yet simple natural-gradient updates by
using a duality associated with exponential-family distributions. An attractive
feature of these methods is that, by using natural-gradients, they are able to
extract accurate local approximations for individual model components. We
summarize recent results for Bayesian deep learning showing the superiority of
natural-gradient approaches over their gradient counterparts.
Competitive Analysis System for Theatrical Movie Releases Based on Movie Trailer Deep Video Representation
Authors: Miguel Campo, Cheng-Kang Hsieh, Matt Nickens, JJ Espinoza, Abhinav Taliyan, Julie Rieger, Jean Ho, Bettina Sherick
Date: 2018-07-12 00:00:00
Tasks: Collaborative Filtering

Audience discovery is an important activity at major movie studios. Deep
models that use convolutional networks to extract frame-by-frame features of a
movie trailer and represent it in a form that is suitable for prediction are
now possible thanks to the availability of pre-built feature extractors trained
on large image datasets. Using these pre-built feature extractors, we are able
to process hundreds of publicly available movie trailers, extract
frame-by-frame low level features (e.g., a face, an object, etc) and create
video-level representations. We use the video-level representations to train a
hybrid Collaborative Filtering model that combines video features with
historical movie attendance records. The trained model not only makes accurate
attendance and audience prediction for existing movies, but also successfully
profiles new movies six to eight months prior to their release.