pwc-archive/papers-with-abstracts huggingface dataset:
A Deep Learning Approach for Blind Drift Calibration of Sensor Networks
Authors: Yuzhi Wang, Anqi Yang, Xiaoming Chen, Pengjun Wang, Yu Wang, Huazhong Yang
Date: 2017-06-16 00:00:00

Temporal drift of sensory data is a severe problem impacting the data quality
of wireless sensor networks (WSNs). With the proliferation of large-scale and
long-term WSNs, it is becoming more important to calibrate sensors when the
ground truth is unavailable. This problem is called "blind calibration". In
this paper, we propose a novel deep learning method named projection-recovery
network (PRNet) to blindly calibrate sensor measurements online. The PRNet
first projects the drifted data to a feature space, and uses a powerful deep
convolutional neural network to recover the estimated drift-free measurements.
We deploy a 24-sensor testbed and provide comprehensive empirical evidence
showing that the proposed method significantly improves the sensing accuracy
and drifted sensor detection. Compared with previous methods, PRNet can
calibrate 2x of drifted sensors at the recovery rate of 80% under the same
level of accuracy requirement. We also provide helpful insights for designing
deep neural networks for sensor calibration. We hope our proposed simple and
effective approach will serve as a solid baseline in blind drift calibration of
sensor networks.
L2 Regularization versus Batch and Weight Normalization
Authors: Twan van Laarhoven
Date: 2017-06-16 00:00:00
Tasks: L2 Regularization

Batch Normalization is a commonly used trick to improve the training of deep
neural networks. These neural networks use L2 regularization, also called
weight decay, ostensibly to prevent overfitting. However, we show that L2
regularization has no regularizing effect when combined with normalization.
Instead, regularization has an influence on the scale of weights, and thereby
on the effective learning rate. We investigate this dependence, both in theory,
and experimentally. We show that popular optimization methods such as ADAM only
partially eliminate the influence of normalization on the learning rate. This
leads to a discussion on other ways to mitigate this issue.
Adversarial-Playground: A Visualization Suite for Adversarial Sample Generation
Authors: Andrew Norton, Yanjun Qi
Date: 2017-06-06 00:00:00
Tasks: BIG-bench Machine Learning

With growing interest in adversarial machine learning, it is important for
machine learning practitioners and users to understand how their models may be
attacked. We propose a web-based visualization tool, Adversarial-Playground, to
demonstrate the efficacy of common adversarial methods against a deep neural
network (DNN) model, built on top of the TensorFlow library.
Adversarial-Playground provides users an efficient and effective experience in
exploring techniques generating adversarial examples, which are inputs crafted
by an adversary to fool a machine learning system. To enable
Adversarial-Playground to generate quick and accurate responses for users, we
use two primary tactics: (1) We propose a faster variant of the
state-of-the-art Jacobian saliency map approach that maintains a comparable
evasion rate. (2) Our visualization does not transmit the generated adversarial
images to the client, but rather only the matrix describing the sample and the
vector representing classification likelihoods.
  The source code along with the data from all of our experiments are available
at \url{https://github.com/QData/AdversarialDNN-Playground}.
Robotic Ironing with 3D Perception and Force/Torque Feedback in Household Environments
Authors: David Estevez, Juan G. Victores, Raul Fernandez-Fernandez, Carlos Balaguer
Date: 2017-06-16 00:00:00

As robotic systems become more popular in household environments, the
complexity of required tasks also increases. In this work we focus on a
domestic chore deemed dull by a majority of the population, the task of
ironing. The presented algorithm improves on the limited number of previous
works by joining 3D perception with force/torque sensing, with emphasis on
finding a practical solution with a feasible implementation in a domestic
setting. Our algorithm obtains a point cloud representation of the working
environment. From this point cloud, the garment is segmented and a custom
Wrinkleness Local Descriptor (WiLD) is computed to determine the location of
the present wrinkles. Using this descriptor, the most suitable ironing path is
computed and, based on it, the manipulation algorithm performs the
force-controlled ironing operation. Experiments have been performed with a
humanoid robot platform, proving that our algorithm is able to detect
successfully wrinkles present in garments and iteratively reduce the
wrinkleness using an unmodified iron.
Deep Structured Features for Semantic Segmentation
Authors: Michael Tschannen, Lukas Cavigelli, Fabian Mentzer, Thomas Wiatowski, Luca Benini
Date: 2016-09-26 00:00:00
Tasks: General Classification, Segmentation, Semantic Segmentation

We propose a highly structured neural network architecture for semantic
segmentation with an extremely small model size, suitable for low-power
embedded and mobile platforms. Specifically, our architecture combines i) a
Haar wavelet-based tree-like convolutional neural network (CNN), ii) a random
layer realizing a radial basis function kernel approximation, and iii) a linear
classifier. While stages i) and ii) are completely pre-specified, only the
linear classifier is learned from data. We apply the proposed architecture to
outdoor scene and aerial image semantic segmentation and show that the accuracy
of our architecture is competitive with conventional pixel classification CNNs.
Furthermore, we demonstrate that the proposed architecture is data efficient in
the sense of matching the accuracy of pixel classification CNNs when trained on
a much smaller data set.
Value-Decomposition Networks For Cooperative Multi-Agent Learning
Authors: Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, Nicolas Sonnerat, Joel Z. Leibo, Karl Tuyls, Thore Graepel
Date: 2017-06-16 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), SMAC+

We study the problem of cooperative multi-agent reinforcement learning with a
single joint reward signal. This class of learning problems is difficult
because of the often large combined action and observation spaces. In the fully
centralized and decentralized approaches, we find the problem of spurious
rewards and a phenomenon we call the "lazy agent" problem, which arises due to
partial observability. We address these problems by training individual agents
with a novel value decomposition network architecture, which learns to
decompose the team value function into agent-wise value functions. We perform
an experimental evaluation across a range of partially-observable multi-agent
domains and show that learning such value-decompositions leads to superior
results, in particular when combined with weight sharing, role information and
information channels.
Coupling Distributed and Symbolic Execution for Natural Language Queries
Authors: Lili Mou, Zhengdong Lu, Hang Li, Zhi Jin
Date: 2016-12-08 00:00:00
Proceeding: ICML 2017 8
Tasks: Natural Language Queries

Building neural networks to query a knowledge base (a table) with natural
language is an emerging research topic in deep learning. An executor for table
querying typically requires multiple steps of execution because queries may
have complicated structures. In previous studies, researchers have developed
either fully distributed executors or symbolic executors for table querying. A
distributed executor can be trained in an end-to-end fashion, but is weak in
terms of execution efficiency and explicit interpretability. A symbolic
executor is efficient in execution, but is very difficult to train especially
at initial stages. In this paper, we propose to couple distributed and symbolic
execution for natural language queries, where the symbolic executor is
pretrained with the distributed executor's intermediate execution results in a
step-by-step fashion. Experiments show that our approach significantly
outperforms both distributed and symbolic executors, exhibiting high accuracy,
high learning efficiency, high execution efficiency, and high interpretability.
Deep Active Learning for Dialogue Generation
Authors: Nabiha Asghar, Pascal Poupart, Xin Jiang, Hang Li
Date: 2016-12-12 00:00:00
Proceeding: SEMEVAL 2017 8
Tasks: Active Learning, Dialogue Generation, Response Generation

We propose an online, end-to-end, neural generative conversational model for
open-domain dialogue. It is trained using a unique combination of offline
two-phase supervised learning and online human-in-the-loop active learning.
While most existing research proposes offline supervision or hand-crafted
reward functions for online reinforcement, we devise a novel interactive
learning mechanism based on hamming-diverse beam search for response generation
and one-character user-feedback at each step. Experiments show that our model
inherently promotes the generation of semantically relevant and interesting
responses, and can be used to train agents with customized personas, moods and
conversational styles.
From Propositional Logic to Plausible Reasoning: A Uniqueness Theorem
Authors: Kevin S. Van Horn
Date: 2017-06-16 00:00:00

We consider the question of extending propositional logic to a logic of
plausible reasoning, and posit four requirements that any such extension should
satisfy. Each is a requirement that some property of classical propositional
logic be preserved in the extended logic; as such, the requirements are simpler
and less problematic than those used in Cox's Theorem and its variants. As with
Cox's Theorem, our requirements imply that the extended logic must be
isomorphic to (finite-set) probability theory. We also obtain specific
numerical values for the probabilities, recovering the classical definition of
probability as a theorem, with truth assignments that satisfy the premise
playing the role of the "possible cases."
Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network
Authors: Frosti Palsson, Johannes R. Sveinsson, Magnus O. Ulfarsson
Date: 2017-06-16 00:00:00
Tasks: Dimensionality Reduction

In this paper, we propose a method using a three dimensional convolutional
neural network (3-D-CNN) to fuse together multispectral (MS) and hyperspectral
(HS) images to obtain a high resolution hyperspectral image. Dimensionality
reduction of the hyperspectral image is performed prior to fusion in order to
significantly reduce the computational time and make the method more robust to
noise. Experiments are performed on a data set simulated using a real
hyperspectral image. The results obtained show that the proposed approach is
very promising when compared to conventional methods. This is especially true
when the hyperspectral image is corrupted by additive noise.
Self-adaptive node-based PCA encodings
Authors: Leonard Johard, Victor Rivera, Manuel Mazzara, Jooyoung Lee
Date: 2017-06-16 00:00:00

In this paper we propose an algorithm, Simple Hebbian PCA, and prove that it
is able to calculate the principal component analysis (PCA) in a distributed
fashion across nodes. It simplifies existing network structures by removing
intralayer weights, essentially cutting the number of weights that need to be
trained in half.
Measuring Neural Net Robustness with Constraints
Authors: Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, Antonio Criminisi
Date: 2016-05-24 00:00:00
Proceeding: NeurIPS 2016 12

Despite having high accuracy, neural nets have been shown to be susceptible
to adversarial examples, where a small perturbation to an input can cause it to
become mislabeled. We propose metrics for measuring the robustness of a neural
net and devise a novel algorithm for approximating these metrics based on an
encoding of robustness as a linear program. We show how our metrics can be used
to evaluate the robustness of deep neural nets with experiments on the MNIST
and CIFAR-10 datasets. Our algorithm generates more informative estimates of
robustness metrics compared to estimates based on existing algorithms.
Furthermore, we show how existing approaches to improving robustness "overfit"
to adversarial examples generated using a specific algorithm. Finally, we show
that our techniques can be used to additionally improve neural net robustness
both according to the metrics that we propose, but also according to previously
proposed metrics.
Semi-Supervised Classification Based on Classification from Positive and Unlabeled Data
Authors: Tomoya Sakai, Marthinus Christoffel du Plessis, Gang Niu, Masashi Sugiyama
Date: 2016-05-23 00:00:00
Proceeding: ICML 2017 8
Tasks: Classification, General Classification

Most of the semi-supervised classification methods developed so far use
unlabeled data for regularization purposes under particular distributional
assumptions such as the cluster assumption. In contrast, recently developed
methods of classification from positive and unlabeled data (PU classification)
use unlabeled data for risk evaluation, i.e., label information is directly
extracted from unlabeled data. In this paper, we extend PU classification to
also incorporate negative data and propose a novel semi-supervised
classification approach. We establish generalization error bounds for our novel
methods and show that the bounds decrease with respect to the number of
unlabeled data without the distributional assumptions that are required in
existing semi-supervised classification methods. Through experiments, we
demonstrate the usefulness of the proposed methods.
Two-pixel polarimetric camera by compressive sensing
Authors: Julien Fade, Estéban Perrotin, Jérôme Bobin
Date: 2017-06-16 00:00:00
Tasks: Compressive Sensing, Vocal Bursts Valence Prediction

We propose an original concept of compressive sensing (CS) polarimetric
imaging based on a digital micro-mirror (DMD) array and two single-pixel
detectors. The polarimetric sensitivity of the proposed setup is due to an
experimental imperfection of reflecting mirrors which is exploited here to form
an original reconstruction problem, including a CS problem and a source
separation task. We show that a two-step approach tackling each problem
successively is outperformed by a dedicated combined reconstruction method,
which is explicited in this article and preferably implemented through a
reweighted FISTA algorithm. The combined reconstruction approach is then
further improved by including physical constraints specific to the polarimetric
imaging context considered, which are implemented in an original constrained
GFB algorithm. Numerical simulations demonstrate the efficiency of the 2-pixel
CS polarimetric imaging setup to retrieve polarimetric contrast data with
significant compression rate and good reconstruction quality. The influence of
experimental imperfections of the DMD are also analyzed through numerical
simulations, and 2D polarimetric imaging reconstruction results are finally
presented.
Encoding of phonology in a recurrent neural model of grounded speech
Authors: Afra Alishahi, Marie Barking, Grzegorz Chrupała
Date: 2017-06-12 00:00:00
Proceeding: CONLL 2017 8
Tasks: Clustering

We study the representation and encoding of phonemes in a recurrent neural
network model of grounded speech. We use a model which processes images and
their spoken descriptions, and projects the visual and auditory representations
into the same semantic space. We perform a number of analyses on how
information about individual phonemes is encoded in the MFCC features extracted
from the speech signal, and the activations of the layers of the model. Via
experiments with phoneme decoding and phoneme discrimination we show that
phoneme representations are most salient in the lower layers of the model,
where low-level signals are processed at a fine-grained level, although a large
amount of phonological information is retain at the top recurrent layer. We
further find out that the attention mechanism following the top recurrent layer
significantly attenuates encoding of phonology and makes the utterance
embeddings much more invariant to synonymy. Moreover, a hierarchical clustering
of phoneme representations learned by the network shows an organizational
structure of phonemes similar to those proposed in linguistics.
Improving Scalability of Inductive Logic Programming via Pruning and Best-Effort Optimisation
Authors: Mishal Kazmi, Peter Schüller, Yücel Saygın
Date: 2017-06-16 00:00:00
Tasks: Chunking, Inductive logic programming, Sentence

Inductive Logic Programming (ILP) combines rule-based and statistical
artificial intelligence methods, by learning a hypothesis comprising a set of
rules given background knowledge and constraints for the search space. We focus
on extending the XHAIL algorithm for ILP which is based on Answer Set
Programming and we evaluate our extensions using the Natural Language
Processing application of sentence chunking. With respect to processing natural
language, ILP can cater for the constant change in how we use language on a
daily basis. At the same time, ILP does not require huge amounts of training
examples such as other statistical methods and produces interpretable results,
that means a set of rules, which can be analysed and tweaked if necessary. As
contributions we extend XHAIL with (i) a pruning mechanism within the
hypothesis generalisation algorithm which enables learning from larger
datasets, (ii) a better usage of modern solver technology using recently
developed optimisation methods, and (iii) a time budget that permits the usage
of suboptimal results. We evaluate these improvements on the task of sentence
chunking using three datasets from a recent SemEval competition. Results show
that our improvements allow for learning on bigger datasets with results that
are of similar quality to state-of-the-art systems on the same task. Moreover,
we compare the hypotheses obtained on datasets to gain insights on the
structure of each dataset.
A Fully Trainable Network with RNN-based Pooling
Authors: Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, Yanbo Gao
Date: 2017-06-16 00:00:00

Pooling is an important component in convolutional neural networks (CNNs) for
aggregating features and reducing computational burden. Compared with other
components such as convolutional layers and fully connected layers which are
completely learned from data, the pooling component is still handcrafted such
as max pooling and average pooling. This paper proposes a learnable pooling
function using recurrent neural networks (RNN) so that the pooling can be fully
adapted to data and other components of the network, leading to an improved
performance. Such a network with learnable pooling function is referred to as a
fully trainable network (FTN). Experimental results have demonstrated that the
proposed RNN-based pooling can well approximate the existing pooling functions
and improve the performance of the network. Especially for small networks, the
proposed FTN can improve the performance by seven percentage points in terms of
error rate on the CIFAR-10 dataset compared with the traditional CNN.
The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge
Authors: He-Da Wang, Teng Zhang, Ji Wu
Date: 2017-06-16 00:00:00
Tasks: General Classification, Video Classification, Video Understanding

This article describes the final solution of team monkeytyping, who finished
in second place in the YouTube-8M video understanding challenge. The dataset
used in this challenge is a large-scale benchmark for multi-label video
classification. We extend the work in [1] and propose several improvements for
frame sequence modeling. We propose a network structure called Chaining that
can better capture the interactions between labels. Also, we report our
approaches in dealing with multi-scale information and attention pooling. In
addition, We find that using the output of model ensemble as a side target in
training can boost single model performance. We report our experiments in
bagging, boosting, cascade, and stacking, and propose a stacking algorithm
called attention weighted stacking. Our final submission is an ensemble that
consists of 74 sub models, all of which are listed in the appendix.
AI-Powered Social Bots
Authors: Terrence Adams
Date: 2017-06-16 00:00:00
Tasks: BIG-bench Machine Learning

This paper gives an overview of impersonation bots that generate output in
one, or possibly, multiple modalities. We also discuss rapidly advancing areas
of machine learning and artificial intelligence that could lead to
frighteningly powerful new multi-modal social bots. Our main conclusion is that
most commonly known bots are one dimensional (i.e., chatterbot), and far from
deceiving serious interrogators. However, using recent advances in machine
learning, it is possible to unleash incredibly powerful, human-like armies of
social bots, in potentially well coordinated campaigns of deception and
influence.
An Automatic Approach for Document-level Topic Model Evaluation
Authors: Shraey Bhatia, Jey Han Lau, Timothy Baldwin
Date: 2017-06-16 00:00:00
Proceeding: CONLL 2017 8
Tasks: Topic Models

Topic models jointly learn topics and document-level topic distribution.
Extrinsic evaluation of topic models tends to focus exclusively on topic-level
evaluation, e.g. by assessing the coherence of topics. We demonstrate that
there can be large discrepancies between topic- and document-level model
quality, and that basing model evaluation on topic-level analysis can be highly
misleading. We propose a method for automatically predicting topic model
quality based on analysis of document-level topic allocations, and provide
empirical evidence for its robustness.
One Model To Learn Them All
Authors: Lukasz Kaiser, Aidan N. Gomez, Noam Shazeer, Ashish Vaswani, Niki Parmar, Llion Jones, Jakob Uszkoreit
Date: 2017-06-16 00:00:00
Tasks: All, Image Captioning, image-classification, Image Classification, model, Multi-Task Learning, speech-recognition, Translation

Deep learning yields great results across many fields, from speech
recognition, image classification, to translation. But for each problem,
getting a deep model to work well involves research into the architecture and a
long period of tuning. We present a single model that yields good results on a
number of problems spanning multiple domains. In particular, this single model
is trained concurrently on ImageNet, multiple translation tasks, image
captioning (COCO dataset), a speech recognition corpus, and an English parsing
task. Our model architecture incorporates building blocks from multiple
domains. It contains convolutional layers, an attention mechanism, and
sparsely-gated layers. Each of these computational blocks is crucial for a
subset of the tasks we train on. Interestingly, even if a block is not crucial
for a task, we observe that adding it never hurts performance and in most cases
improves it on all tasks. We also show that tasks with less data benefit
largely from joint training with other tasks, while performance on large tasks
degrades only slightly if at all.