pwc-archive/papers-with-abstracts huggingface dataset:
Be Careful What You Backpropagate: A Case For Linear Output Activations & Gradient Boosting
Authors: Anders Oland, Aayush Bansal, Roger B. Dannenberg, Bhiksha Raj
Date: 2017-07-13 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Semantic Segmentation

In this work, we show that saturating output activation functions, such as
the softmax, impede learning on a number of standard classification tasks.
Moreover, we present results showing that the utility of softmax does not stem
from the normalization, as some have speculated. In fact, the normalization
makes things worse. Rather, the advantage is in the exponentiation of error
gradients. This exponential gradient boosting is shown to speed up convergence
and improve generalization. To this end, we demonstrate faster convergence and
better performance on diverse classification tasks: image classification using
CIFAR-10 and ImageNet, and semantic segmentation using PASCAL VOC 2012. In the
latter case, using the state-of-the-art neural network architecture, the model
converged 33% faster with our method (roughly two days of training less) than
with the standard softmax activation, and with a slightly better performance to
boot.
Generalising Random Forest Parameter Optimisation to Include Stability and Cost
Authors: C. H. Bryan Liu, Benjamin Paul Chamberlain, Duncan A. Little, Angelo Cardoso
Date: 2017-06-29 00:00:00
Tasks: Bayesian Optimisation

Random forests are among the most popular classification and regression
methods used in industrial applications. To be effective, the parameters of
random forests must be carefully tuned. This is usually done by choosing values
that minimize the prediction error on a held out dataset. We argue that error
reduction is only one of several metrics that must be considered when
optimizing random forest parameters for commercial applications. We propose a
novel metric that captures the stability of random forests predictions, which
we argue is key for scenarios that require successive predictions. We motivate
the need for multi-criteria optimization by showing that in practical
applications, simply choosing the parameters that lead to the lowest error can
introduce unnecessary costs and produce predictions that are not stable across
independent runs. To optimize this multi-criteria trade-off, we present a new
framework that efficiently finds a principled balance between these three
considerations using Bayesian optimisation. The pitfalls of optimising forest
parameters purely for error reduction are demonstrated using two publicly
available real world datasets. We show that our framework leads to parameter
settings that are markedly different from the values discovered by error
reduction metrics.
Distral: Robust Multitask Reinforcement Learning
Authors: Yee Whye Teh, Victor Bapst, Wojciech Marian Czarnecki, John Quan, James Kirkpatrick, Raia Hadsell, Nicolas Heess, Razvan Pascanu
Date: 2017-07-13 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Transfer Learning

Most deep reinforcement learning algorithms are data inefficient in complex
and rich environments, limiting their applicability to many scenarios. One
direction for improving data efficiency is multitask learning with shared
neural network parameters, where efficiency may be improved through transfer
across related tasks. In practice, however, this is not usually observed,
because gradients from different tasks can interfere negatively, making
learning unstable and sometimes even less data efficient. Another issue is the
different reward schemes between tasks, which can easily lead to one task
dominating the learning of a shared model. We propose a new approach for joint
training of multiple tasks, which we refer to as Distral (Distill & transfer
learning). Instead of sharing parameters between the different workers, we
propose to share a "distilled" policy that captures common behaviour across
tasks. Each worker is trained to solve its own task while constrained to stay
close to the shared policy, while the shared policy is trained by distillation
to be the centroid of all task policies. Both aspects of the learning process
are derived by optimizing a joint objective function. We show that our approach
supports efficient transfer on complex 3D environments, outperforming several
related methods. Moreover, the proposed learning process is more robust and
more stable---attributes that are critical in deep reinforcement learning.
Exponential scaling of neural algorithms - a future beyond Moore's Law?
Authors: James B. Aimone
Date: 2017-05-04 00:00:00

Although the brain has long been considered a potential inspiration for
future computing, Moore's Law - the scaling property that has seen revolutions
in technologies ranging from supercomputers to smart phones - has largely been
driven by advances in materials science. As the ability to miniaturize
transistors is coming to an end, there is increasing attention on new
approaches to computation, including renewed enthusiasm around the potential of
neural computation. This paper describes how recent advances in
neurotechnologies, many of which have been aided by computing's rapid
progression over recent decades, are now reigniting this opportunity to bring
neural computation insights into broader computing applications. As we
understand more about the brain, our ability to motivate new computing
paradigms with continue to progress. These new approaches to computing, which
we are already seeing in techniques such as deep learning and neuromorphic
hardware, will themselves improve our ability to learn about the brain and
accordingly can be projected to give rise to even further insights. This paper
will describe how this positive feedback has the potential to change the
complexion of how computing sciences and neurosciences interact, and suggests
that the next form of exponential scaling in computing may emerge from our
progressive understanding of the brain.
Structure Learning in Motor Control:A Deep Reinforcement Learning Model
Authors: Ari Weinstein, Matthew M. Botvinick
Date: 2017-06-21 00:00:00
Tasks: Deep Reinforcement Learning, Model-based Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Motor adaptation displays a structure-learning effect: adaptation to a new
perturbation occurs more quickly when the subject has prior exposure to
perturbations with related structure. Although this `learning-to-learn' effect
is well documented, its underlying computational mechanisms are poorly
understood. We present a new model of motor structure learning, approaching it
from the point of view of deep reinforcement learning. Previous work outside of
motor control has shown how recurrent neural networks can account for
learning-to-learn effects. We leverage this insight to address motor learning,
by importing it into the setting of model-based reinforcement learning. We
apply the resulting processing architecture to empirical findings from a
landmark study of structure learning in target-directed reaching (Braun et al.,
2009), and discuss its implications for a wider range of learning-to-learn
phenomena.
UTS submission to Google YouTube-8M Challenge 2017
Authors: Linchao Zhu, Yanbin Liu, Yi Yang
Date: 2017-07-13 00:00:00
Tasks: Classification, General Classification, Mixture-of-Experts, Video Classification

In this paper, we present our solution to Google YouTube-8M Video
Classification Challenge 2017. We leveraged both video-level and frame-level
features in the submission. For video-level classification, we simply used a
200-mixture Mixture of Experts (MoE) layer, which achieves GAP 0.802 on the
validation set with a single model. For frame-level classification, we utilized
several variants of recurrent neural networks, sequence aggregation with
attention mechanism and 1D convolutional models. We achieved GAP 0.8408 on the
private testing set with the ensemble model.
  The source code of our models can be found in
\url{https://github.com/ffmpbgrnn/yt8m}.
Preliminary results on Ontology-based Open Data Publishing
Authors: Gianluca Cima
Date: 2017-05-30 00:00:00
Tasks: Management

Despite the current interest in Open Data publishing, a formal and
comprehensive methodology supporting an organization in deciding which data to
publish and carrying out precise procedures for publishing high-quality data,
is still missing. In this paper we argue that the Ontology-based Data
Management paradigm can provide a formal basis for a principled approach to
publish high quality, semantically annotated Open Data. We describe two main
approaches to using an ontology for this endeavor, and then we present some
technical results on one of the approaches, called bottom-up, where the
specification of the data to be published is given in terms of the sources, and
specific techniques allow deriving suitable annotations for interpreting the
published data under the light of the ontology.
PAC-Bayesian Analysis for a two-step Hierarchical Multiview Learning Approach
Authors: Anil Goyal, Emilie Morvant, Pascal Germain, Massih-Reza Amini
Date: 2016-06-23 00:00:00
Tasks: Diversity, Multiview Learning, Vocal Bursts Valence Prediction

We study a two-level multiview learning with more than two views under the
PAC-Bayesian framework. This approach, sometimes referred as late fusion,
consists in learning sequentially multiple view-specific classifiers at the
first level, and then combining these view-specific classifiers at the second
level. Our main theoretical result is a generalization bound on the risk of the
majority vote which exhibits a term of diversity in the predictions of the
view-specific classifiers. From this result it comes out that controlling the
trade-off between diversity and accuracy is a key element for multiview
learning, which complements other results in multiview learning. Finally, we
experiment our principle on multiview datasets extracted from the Reuters
RCV1/RCV2 collection.
Do Convolutional Networks need to be Deep for Text Classification ?
Authors: Hoa T. Le, Christophe Cerisara, Alexandre Denis
Date: 2017-07-13 00:00:00
Tasks: Classification, General Classification, Sentiment Analysis, text-classification, Text Classification

We study in this work the importance of depth in convolutional models for
text classification, either when character or word inputs are considered. We
show on 5 standard text classification and sentiment analysis tasks that deep
models indeed give better performances than shallow networks when the text
input is represented as a sequence of characters. However, a simple
shallow-and-wide network outperforms deep models such as DenseNet with word
inputs. Our shallow word model further establishes new state-of-the-art
performances on two datasets: Yelp Binary (95.9\%) and Yelp Full (64.9\%).
Knowledge Elicitation via Sequential Probabilistic Inference for High-Dimensional Prediction
Authors: Pedram Daee, Tomi Peltola, Marta Soare, Samuel Kaski
Date: 2016-12-10 00:00:00
Tasks: Prediction, regression, Vocal Bursts Intensity Prediction

Prediction in a small-sized sample with a large number of covariates, the
"small n, large p" problem, is challenging. This setting is encountered in
multiple applications, such as precision medicine, where obtaining additional
samples can be extremely costly or even impossible, and extensive research
effort has recently been dedicated to finding principled solutions for accurate
prediction. However, a valuable source of additional information, domain
experts, has not yet been efficiently exploited. We formulate knowledge
elicitation generally as a probabilistic inference process, where expert
knowledge is sequentially queried to improve predictions. In the specific case
of sparse linear regression, where we assume the expert has knowledge about the
values of the regression coefficients or about the relevance of the features,
we propose an algorithm and computational approximation for fast and efficient
interaction, which sequentially identifies the most informative features on
which to query expert knowledge. Evaluations of our method in experiments with
simulated and real users show improved prediction accuracy already with a small
effort from the expert.
Is writing style predictive of scientific fraud?
Authors: Chloé Braud, Anders Søgaard
Date: 2017-07-13 00:00:00
Tasks: Logical Reasoning

The problem of detecting scientific fraud using machine learning was recently
introduced, with initial, positive results from a model taking into account
various general indicators. The results seem to suggest that writing style is
predictive of scientific fraud. We revisit these initial experiments, and show
that the leave-one-out testing procedure they used likely leads to a slight
over-estimate of the predictability, but also that simple models can outperform
their proposed model by some margin. We go on to explore more abstract
linguistic features, such as linguistic complexity and discourse structure,
only to obtain negative results. Upon analyzing our models, we do see some
interesting patterns, though: Scientific fraud, for examples, contains less
comparison, as well as different types of hedging and ways of presenting
logical reasoning.
Learning like humans with Deep Symbolic Networks
Authors: Qunzhi Zhang, Didier Sornette
Date: 2017-07-11 00:00:00
Tasks: Small Data Image Classification

We introduce the Deep Symbolic Network (DSN) model, which aims at becoming
the white-box version of Deep Neural Networks (DNN). The DSN model provides a
simple, universal yet powerful structure, similar to DNN, to represent any
knowledge of the world, which is transparent to humans. The conjecture behind
the DSN model is that any type of real world objects sharing enough common
features are mapped into human brains as a symbol. Those symbols are connected
by links, representing the composition, correlation, causality, or other
relationships between them, forming a deep, hierarchical symbolic network
structure. Powered by such a structure, the DSN model is expected to learn like
humans, because of its unique characteristics. First, it is universal, using
the same structure to store any knowledge. Second, it can learn symbols from
the world and construct the deep symbolic networks automatically, by utilizing
the fact that real world objects have been naturally separated by
singularities. Third, it is symbolic, with the capacity of performing causal
deduction and generalization. Fourth, the symbols and the links between them
are transparent to us, and thus we will know what it has learned or not - which
is the key for the security of an AI system. Fifth, its transparency enables it
to learn with relatively small data. Sixth, its knowledge can be accumulated.
Last but not least, it is more friendly to unsupervised learning than DNN. We
present the details of the model, the algorithm powering its automatic learning
ability, and describe its usefulness in different use cases. The purpose of
this paper is to generate broad interest to develop it within an open source
project centered on the Deep Symbolic Network (DSN) model towards the
development of general AI.
Automation of Feature Engineering for IoT Analytics
Authors: Snehasis Banerjee, Tanushyam Chattopadhyay, Arpan Pal, Utpal Garain
Date: 2017-07-13 00:00:00
Tasks: Decision Making, Feature Engineering, feature selection, Survey

This paper presents an approach for automation of interpretable feature
selection for Internet Of Things Analytics (IoTA) using machine learning (ML)
techniques. Authors have conducted a survey over different people involved in
different IoTA based application development tasks. The survey reveals that
feature selection is the most time consuming and niche skill demanding part of
the entire workflow. This paper shows how feature selection is successfully
automated without sacrificing the decision making accuracy and thereby reducing
the project completion time and cost of hiring expensive resources. Several
pattern recognition principles and state of art (SoA) ML techniques are
followed to design the overall approach for the proposed automation. Three data
sets are considered to establish the proof-of-concept. Experimental results
show that the proposed automation is able to reduce the time for feature
selection to $2$ days instead of $4-6$ months which would have been required in
absence of the automation. This reduction in time is achieved without any
sacrifice in the accuracy of the decision making process. Proposed method is
also compared against Multi Layer Perceptron (MLP) model as most of the state
of the art works on IoTA uses MLP based Deep Learning. Moreover the feature
selection method is compared against SoA feature reduction technique namely
Principal Component Analysis (PCA) and its variants. The results obtained show
that the proposed method is effective.
Clingo goes Linear Constraints over Reals and Integers
Authors: Tomi Janhunen, Roland Kaminski, Max Ostrowski, Torsten Schaub, Sebastian Schellhorn, Philipp Wanko
Date: 2017-07-13 00:00:00

The recent series 5 of the ASP system clingo provides generic means to
enhance basic Answer Set Programming (ASP) with theory reasoning capabilities.
We instantiate this framework with different forms of linear constraints,
discuss the respective implementations, and present techniques of how to use
these constraints in a reactive context. More precisely, we introduce
extensions to clingo with difference and linear constraints over integers and
reals, respectively, and realize them in complementary ways. Finally, we
empirically evaluate the resulting clingo derivatives clingo[dl] and clingo[lp]
on common fragments and contrast them to related ASP systems.
  This paper is under consideration for acceptance in TPLP.
Discrete Multi-modal Hashing with Canonical Views for Robust Mobile Landmark Search
Authors: Lei Zhu, Zi Huang, Xiaobai Liu, Xiangnan He, Jingkuan Song, Xiaofang Zhou
Date: 2017-07-13 00:00:00

Mobile landmark search (MLS) recently receives increasing attention for its
great practical values. However, it still remains unsolved due to two important
challenges. One is high bandwidth consumption of query transmission, and the
other is the huge visual variations of query images sent from mobile devices.
In this paper, we propose a novel hashing scheme, named as canonical view based
discrete multi-modal hashing (CV-DMH), to handle these problems via a novel
three-stage learning procedure. First, a submodular function is designed to
measure visual representativeness and redundancy of a view set. With it,
canonical views, which capture key visual appearances of landmark with limited
redundancy, are efficiently discovered with an iterative mining strategy.
Second, multi-modal sparse coding is applied to transform visual features from
multiple modalities into an intermediate representation. It can robustly and
adaptively characterize visual contents of varied landmark images with certain
canonical views. Finally, compact binary codes are learned on intermediate
representation within a tailored discrete binary embedding model which
preserves visual relations of images measured with canonical views and removes
the involved noises. In this part, we develop a new augmented Lagrangian
multiplier (ALM) based optimization method to directly solve the discrete
binary codes. We can not only explicitly deal with the discrete constraint, but
also consider the bit-uncorrelated constraint and balance constraint together.
Experiments on real world landmark datasets demonstrate the superior
performance of CV-DMH over several state-of-the-art methods.
Large-scale Video Classification guided by Batch Normalized LSTM Translator
Authors: Jae Hyeon Yoo
Date: 2017-07-13 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification, Sentence, Video Classification, Video Recognition

Youtube-8M dataset enhances the development of large-scale video recognition
technology as ImageNet dataset has encouraged image classification, recognition
and detection of artificial intelligence fields. For this large video dataset,
it is a challenging task to classify a huge amount of multi-labels. By change
of perspective, we propose a novel method by regarding labels as words. In
details, we describe online learning approaches to multi-label video
classification that are guided by deep recurrent neural networks for video to
sentence translator. We designed the translator based on LSTMs and found out
that a stochastic gating before the input of each LSTM cell can help us to
design the structural details. In addition, we adopted batch normalizations
into our models to improve our LSTM models. Since our models are feature
extractors, they can be used with other classifiers. Finally we report improved
validation results of our models on large-scale Youtube-8M datasets and
discussions for the further improvement.
Random Forests versus Neural Networks - What's Best for Camera Localization?
Authors: Daniela Massiceti, Alexander Krull, Eric Brachmann, Carsten Rother, Philip H. S. Torr
Date: 2016-09-19 00:00:00
Tasks: Camera Localization, regression

This work addresses the task of camera localization in a known 3D scene given
a single input RGB image. State-of-the-art approaches accomplish this in two
steps: firstly, regressing for every pixel in the image its 3D scene coordinate
and subsequently, using these coordinates to estimate the final 6D camera pose
via RANSAC. To solve the first step, Random Forests (RFs) are typically used.
On the other hand, Neural Networks (NNs) reign in many dense regression tasks,
but are not test-time efficient. We ask the question: which of the two is best
for camera localization? To address this, we make two method contributions: (1)
a test-time efficient NN architecture which we term a ForestNet that is derived
and initialized from a RF, and (2) a new fully-differentiable robust averaging
technique for regression ensembles which can be trained end-to-end with a NN.
Our experimental findings show that for scene coordinate regression,
traditional NN architectures are superior to test-time efficient RFs and
ForestNets, however, this does not translate to final 6D camera pose accuracy
where RFs and ForestNets perform slightly better. To summarize, our best
method, a ForestNet with a robust average, which has an equivalent fast and
lightweight RF, improves over the state-of-the-art for camera localization on
the 7-Scenes dataset. While this work focuses on scene coordinate regression
for camera localization, our innovations may also be applied to other
continuous regression tasks.
Constraints, Lazy Constraints, or Propagators in ASP Solving: An Empirical Analysis
Authors: Bernardo Cuteri, Carmine Dodaro, Francesco Ricca, Peter Schüller
Date: 2017-07-13 00:00:00

Answer Set Programming (ASP) is a well-established declarative paradigm. One
of the successes of ASP is the availability of efficient systems.
State-of-the-art systems are based on the ground+solve approach. In some
applications this approach is infeasible because the grounding of one or few
constraints is expensive. In this paper, we systematically compare alternative
strategies to avoid the instantiation of problematic constraints, that are
based on custom extensions of the solver. Results on real and synthetic
benchmarks highlight some strengths and weaknesses of the different strategies.
(Under consideration for acceptance in TPLP, ICLP 2017 Special Issue.)