pwc-archive/papers-with-abstracts huggingface dataset:
Wisdom of the crowd from unsupervised dimension reduction
Authors: Lingfei Wang, Tom Michoel
Date: 2017-11-28 00:00:00
Tasks: Decision Making, Dimensionality Reduction

Wisdom of the crowd, the collective intelligence derived from responses of
multiple human or machine individuals to the same questions, can be more
accurate than each individual, and improve social decision-making and
prediction accuracy. This can also integrate multiple programs or datasets,
each as an individual, for the same predictive questions. Crowd wisdom
estimates each individual's independent error level arising from their limited
knowledge, and finds the crowd consensus that minimizes the overall error.
However, previous studies have merely built isolated, problem-specific models
with limited generalizability, and mainly for binary (yes/no) responses. Here
we show with simulation and real-world data that the crowd wisdom problem is
analogous to one-dimensional unsupervised dimension reduction in machine
learning. This provides a natural class of crowd wisdom solutions, such as
principal component analysis and Isomap, which can handle binary and also
continuous responses, like confidence levels, and consequently can be more
accurate than existing solutions. They can even outperform
supervised-learning-based collective intelligence that is calibrated on
historical performance of individuals, e.g. penalized linear regression and
random forest. This study unifies crowd wisdom and unsupervised dimension
reduction, and thereupon introduces a broad range of highly-performing and
widely-applicable crowd wisdom methods. As the costs for data acquisition and
processing rapidly decrease, this study will promote and guide crowd wisdom
applications in the social and natural sciences, including data fusion,
meta-analysis, crowd-sourcing, and committee decision making.
Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations
Authors: Maziar Raissi, Paris Perdikaris, George Em. Karniadakis
Date: 2017-11-28 00:00:00

We introduce physics informed neural networks -- neural networks that are
trained to solve supervised learning tasks while respecting any given law of
physics described by general nonlinear partial differential equations. In this
second part of our two-part treatise, we focus on the problem of data-driven
discovery of partial differential equations. Depending on whether the available
data is scattered in space-time or arranged in fixed temporal snapshots, we
introduce two main classes of algorithms, namely continuous time and discrete
time models. The effectiveness of our approach is demonstrated using a wide
range of benchmark problems in mathematical physics, including conservation
laws, incompressible fluid flow, and the propagation of nonlinear shallow-water
waves.
Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations
Authors: Maziar Raissi, Paris Perdikaris, George Em. Karniadakis
Date: 2017-11-28 00:00:00

We introduce physics informed neural networks -- neural networks that are
trained to solve supervised learning tasks while respecting any given law of
physics described by general nonlinear partial differential equations. In this
two part treatise, we present our developments in the context of solving two
main classes of problems: data-driven solution and data-driven discovery of
partial differential equations. Depending on the nature and arrangement of the
available data, we devise two distinct classes of algorithms, namely continuous
time and discrete time models. The resulting neural networks form a new class
of data-efficient universal function approximators that naturally encode any
underlying physical laws as prior information. In this first part, we
demonstrate how these networks can be used to infer solutions to partial
differential equations, and obtain physics-informed surrogate models that are
fully differentiable with respect to all input coordinates and free parameters.
A Recursive Bayesian Approach To Describe Retinal Vasculature Geometry
Authors: Fatmatulzehra Uslu, Anil Anthony Bharath
Date: 2017-11-28 00:00:00

Demographic studies suggest that changes in the retinal vasculature geometry,
especially in vessel width, are associated with the incidence or progression of
eye-related or systemic diseases. To date, the main information source for
width estimation from fundus images has been the intensity profile between
vessel edges. However, there are many factors affecting the intensity profile:
pathologies, the central light reflex and local illumination levels, to name a
few. In this study, we introduce three information sources for width
estimation. These are the probability profiles of vessel interior, centreline
and edge locations generated by a deep network. The probability profiles
provide direct access to vessel geometry and are used in the likelihood
calculation for a Bayesian method, particle filtering. We also introduce a
geometric model which can handle non-ideal conditions of the probability
profiles. Our experiments conducted on the REVIEW dataset yielded consistent
estimates of vessel width, even in cases when one of the vessel edges is
difficult to identify. Moreover, our results suggest that the method is better
than human observers at locating edges of low contrast vessels.
Highlighting objects of interest in an image by integrating saliency and depth
Authors: Subhayan Mukherjee, Irene Cheng, Anup Basu
Date: 2017-11-28 00:00:00
Tasks: 3D Reconstruction

Stereo images have been captured primarily for 3D reconstruction in the past.
However, the depth information acquired from stereo can also be used along with
saliency to highlight certain objects in a scene. This approach can be used to
make still images more interesting to look at, and highlight objects of
interest in the scene. We introduce this novel direction in this paper, and
discuss the theoretical framework behind the approach. Even though we use depth
from stereo in this work, our approach is applicable to depth data acquired
from any sensor modality. Experimental results on both indoor and outdoor
scenes demonstrate the benefits of our algorithm.
Large-Scale Mapping of Human Activity using Geo-Tagged Videos
Authors: Yi Zhu, Sen Liu, Shawn Newsam
Date: 2017-06-24 00:00:00

This paper is the first work to perform spatio-temporal mapping of human
activity using the visual content of geo-tagged videos. We utilize a recent
deep-learning based video analysis framework, termed hidden two-stream
networks, to recognize a range of activities in YouTube videos. This framework
is efficient and can run in real time or faster which is important for
recognizing events as they occur in streaming video or for reducing latency in
analyzing already captured video. This is, in turn, important for using video
in smart-city applications. We perform a series of experiments to show our
approach is able to accurately map activities both spatially and temporally. We
also demonstrate the advantages of using the visual content over the
tags/titles.
From Parity to Preference-based Notions of Fairness in Classification
Authors: Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, Krishna P. Gummadi, Adrian Weller
Date: 2017-06-30 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Classification, Decision Making, Fairness, General Classification

The adoption of automated, data-driven decision making in an ever expanding
range of applications has raised concerns about its potential unfairness
towards certain social groups. In this context, a number of recent studies have
focused on defining, detecting, and removing unfairness from data-driven
decision systems. However, the existing notions of fairness, based on parity
(equality) in treatment or outcomes for different social groups, tend to be
quite stringent, limiting the overall decision making accuracy. In this paper,
we draw inspiration from the fair-division and envy-freeness literature in
economics and game theory and propose preference-based notions of fairness --
given the choice between various sets of decision treatments or outcomes, any
group of users would collectively prefer its treatment or outcomes, regardless
of the (dis)parity as compared to the other groups. Then, we introduce
tractable proxies to design margin-based classifiers that satisfy these
preference-based notions of fairness. Finally, we experiment with a variety of
synthetic and real-world datasets and show that preference-based fairness
allows for greater decision accuracy than parity-based fairness.
AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks
Authors: Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, Xiaodong He
Date: 2017-11-28 00:00:00
Proceeding: CVPR 2018 6
Tasks: Generative Adversarial Network, Image Generation, Image-text matching, Text Matching, Text to Image Generation, Text-to-Image Generation

In this paper, we propose an Attentional Generative Adversarial Network
(AttnGAN) that allows attention-driven, multi-stage refinement for fine-grained
text-to-image generation. With a novel attentional generative network, the
AttnGAN can synthesize fine-grained details at different subregions of the
image by paying attentions to the relevant words in the natural language
description. In addition, a deep attentional multimodal similarity model is
proposed to compute a fine-grained image-text matching loss for training the
generator. The proposed AttnGAN significantly outperforms the previous state of
the art, boosting the best reported inception score by 14.14% on the CUB
dataset and 170.25% on the more challenging COCO dataset. A detailed analysis
is also performed by visualizing the attention layers of the AttnGAN. It for
the first time shows that the layered attentional GAN is able to automatically
select the condition at the word level for generating different parts of the
image.
Plan, Attend, Generate: Planning for Sequence-to-Sequence Models
Authors: Francis Dutil, Caglar Gulcehre, Adam Trischler, Yoshua Bengio
Date: 2017-11-28 00:00:00
Proceeding: NeurIPS 2017 12
Tasks: Question Generation, Question-Generation, Reinforcement Learning, Reinforcement Learning (RL), Translation

We investigate the integration of a planning mechanism into
sequence-to-sequence models using attention. We develop a model which can plan
ahead in the future when it computes its alignments between input and output
sequences, constructing a matrix of proposed future alignments and a commitment
vector that governs whether to follow or recompute the plan. This mechanism is
inspired by the recently proposed strategic attentive reader and writer (STRAW)
model for Reinforcement Learning. Our proposed model is end-to-end trainable
using primarily differentiable operations. We show that it outperforms a strong
baseline on character-level translation tasks from WMT'15, the algorithmic task
of finding Eulerian circuits of graphs, and question generation from the text.
Our analysis demonstrates that the model computes qualitatively intuitive
alignments, converges faster than the baselines, and achieves superior
performance with fewer parameters.
La production de nitrites lors de la dénitrification des eaux usées par biofiltration - Stratégie de contrôle et de réduction des concentrations résiduelles
Authors: Vincent Rocher, Cédric Join, Stéphane Mottelet, Jean Bernier, Sabrina Rechdaoui-Guérin, Sam Azimi, Paul Lessard, André Pauss, Michel Fliess
Date: 2017-11-28 00:00:00

The recent popularity of post-denitrification processes in the greater Paris
area wastewater treatment plants has caused a resurgence of the presence of
nitrite in the Seine river. Controlling the production of nitrite during the
post-denitrification has thus become a major technical issue. Research studies
have been led in the MOCOPEE program (www.mocopee.com) to better understand the
underlying mechanisms behind the production of nitrite during wastewater
denitrification and to develop technical tools (measurement and control
solutions) to assist on-site reductions of nitrite productions. Prior studies
have shown that typical methanol dosage strategies produce a varying
carbon-to-nitrogen ratio in the reactor, which in turn leads to unstable
nitrite concentrations in the effluent. The possibility of adding a model-free
control to the actual classical dosage strategy has thus been tested on the
SimBio model, which simulates the behavior of wastewater biofilters. The
corresponding "intelligent" feedback loop, which is using effluent nitrite
concentrations, compensates the classical strategy only when needed. Simulation
results show a clear improvement in average nitrite concentration level and
level stability in the effluent, without a notable overcost in methanol.
Variants of RMSProp and Adagrad with Logarithmic Regret Bounds
Authors: Mahesh Chandra Mukkamala, Matthias Hein
Date: 2017-06-17 00:00:00
Proceeding: ICML 2017 8

Adaptive gradient methods have become recently very popular, in particular as
they have been shown to be useful in the training of deep neural networks. In
this paper we have analyzed RMSProp, originally proposed for the training of
deep neural networks, in the context of online convex optimization and show
$\sqrt{T}$-type regret bounds. Moreover, we propose two variants SC-Adagrad and
SC-RMSProp for which we show logarithmic regret bounds for strongly convex
functions. Finally, we demonstrate in the experiments that these new variants
outperform other adaptive gradient techniques or stochastic gradient descent in
the optimization of strongly convex functions as well as in training of deep
neural networks.
Automatically Extracting Action Graphs from Materials Science Synthesis Procedures
Authors: Sheshera Mysore, Edward Kim, Emma Strubell, Ao Liu, Haw-Shiuan Chang, Srikrishna Kompella, Kevin Huang, Andrew McCallum, Elsa Olivetti
Date: 2017-11-18 00:00:00
Tasks: Articles

Computational synthesis planning approaches have achieved recent success in
organic chemistry, where tabulated synthesis procedures are readily available
for supervised learning. The syntheses of inorganic materials, however, exist
primarily as natural language narratives contained within scientific journal
articles. This synthesis information must first be extracted from the text in
order to enable analogous synthesis planning methods for inorganic materials.
In this work, we present a system for automatically extracting structured
representations of synthesis procedures from the texts of materials science
journal articles that describe explicit, experimental syntheses of inorganic
compounds. We define the structured representation as a set of linked events
made up of extracted scientific entities and evaluate two unsupervised
approaches for extracting these structures on expert-annotated articles: a
strong heuristic baseline and a generative model of procedural text. We also
evaluate a variety of supervised models for extracting scientific entities. Our
results provide insight into the nature of the data and directions for further
work in this exciting new area of research.
Accelerated Gradient Descent Escapes Saddle Points Faster than Gradient Descent
Authors: Chi Jin, Praneeth Netrapalli, Michael. I. Jordan
Date: 2017-11-28 00:00:00

Nesterov's accelerated gradient descent (AGD), an instance of the general
family of "momentum methods", provably achieves faster convergence rate than
gradient descent (GD) in the convex setting. However, whether these methods are
superior to GD in the nonconvex setting remains open. This paper studies a
simple variant of AGD, and shows that it escapes saddle points and finds a
second-order stationary point in $\tilde{O}(1/\epsilon^{7/4})$ iterations,
faster than the $\tilde{O}(1/\epsilon^{2})$ iterations required by GD. To the
best of our knowledge, this is the first Hessian-free algorithm to find a
second-order stationary point faster than GD, and also the first single-loop
algorithm with a faster rate than GD even in the setting of finding a
first-order stationary point. Our analysis is based on two key ideas: (1) the
use of a simple Hamiltonian function, inspired by a continuous-time
perspective, which AGD monotonically decreases per step even for nonconvex
functions, and (2) a novel framework called improve or localize, which is
useful for tracking the long-term behavior of gradient-based optimization
algorithms. We believe that these techniques may deepen our understanding of
both acceleration algorithms and nonconvex optimization.
Multi-class Semantic Segmentation of Skin Lesions via Fully Convolutional Networks
Authors: Manu Goyal, Moi Hoon Yap, Saeed Hassanpour
Date: 2017-11-28 00:00:00
Tasks: Segmentation, Semantic Segmentation, Transfer Learning

Melanoma is clinically difficult to distinguish from common benign skin lesions, particularly melanocytic naevus and seborrhoeic keratosis. The dermoscopic appearance of these lesions has huge intra-class variations and high inter-class visual similarities. Most current research is focusing on single-class segmentation irrespective of classes of skin lesions. In this work, we evaluate the performance of deep learning on multi-class segmentation of ISIC-2017 challenge dataset, which consists of 2,750 dermoscopic images. We propose an end-to-end solution using fully convolutional networks (FCNs) for multi-class semantic segmentation to automatically segment the melanoma, seborrhoeic keratosis and naevus. To improve the performance of FCNs, transfer learning and a hybrid loss function are used. We evaluate the performance of the deep learning segmentation methods for multi-class segmentation and lesion diagnosis (with post-processing method) on the testing set of the ISIC-2017 challenge dataset. The results showed that the two-tier level transfer learning FCN-8s achieved the overall best result with \textit{Dice} score of 78.5% in a naevus category, 65.3% in melanoma, and 55.7% in seborrhoeic keratosis in multi-class segmentation and Accuracy of 84.62% for recognition of melanoma in lesion diagnosis.
Shortcut-Stacked Sentence Encoders for Multi-Domain Inference
Authors: Yixin Nie, Mohit Bansal
Date: 2017-08-07 00:00:00
Proceeding: WS 2017 9
Tasks: Natural Language Inference, Sentence, Word Embeddings

We present a simple sequential sentence encoder for multi-domain natural
language inference. Our encoder is based on stacked bidirectional LSTM-RNNs
with shortcut connections and fine-tuning of word embeddings. The overall
supervised model uses the above encoder to encode two input sentences into two
vectors, and then uses a classifier over the vector combination to label the
relationship between these two sentences as that of entailment, contradiction,
or neural. Our Shortcut-Stacked sentence encoders achieve strong improvements
over existing encoders on matched and mismatched multi-domain natural language
inference (top non-ensemble single-model result in the EMNLP RepEval 2017
Shared Task (Nangia et al., 2017)). Moreover, they achieve the new
state-of-the-art encoding result on the original SNLI dataset (Bowman et al.,
2015).
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Authors: Aaron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George van den Driessche, Edward Lockhart, Luis C. Cobo, Florian Stimberg, Norman Casagrande, Dominik Grewe, Seb Noury, Sander Dieleman, Erich Elsen, Nal Kalchbrenner, Heiga Zen, Alex Graves, Helen King, Tom Walters, Dan Belov, Demis Hassabis
Date: 2017-11-28 00:00:00
Proceeding: ICML 2018 7
Tasks: Speech Synthesis, Vocal Bursts Intensity Prediction

The recently-developed WaveNet architecture is the current state of the art
in realistic speech synthesis, consistently rated as more natural sounding for
many different languages than any previous system. However, because WaveNet
relies on sequential generation of one audio sample at a time, it is poorly
suited to today's massively parallel computers, and therefore hard to deploy in
a real-time production setting. This paper introduces Probability Density
Distillation, a new method for training a parallel feed-forward network from a
trained WaveNet with no significant difference in quality. The resulting system
is capable of generating high-fidelity speech samples at more than 20 times
faster than real-time, and is deployed online by Google Assistant, including
serving multiple English and Japanese voices.
AI Safety Gridworlds
Authors: Jan Leike, Miljan Martic, Victoria Krakovna, Pedro A. Ortega, Tom Everitt, Andrew Lefrancq, Laurent Orseau, Shane Legg
Date: 2017-11-27 00:00:00
Tasks: Deep Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Safe Exploration

We present a suite of reinforcement learning environments illustrating
various safety properties of intelligent agents. These problems include safe
interruptibility, avoiding side effects, absent supervisor, reward gaming, safe
exploration, as well as robustness to self-modification, distributional shift,
and adversaries. To measure compliance with the intended safe behavior, we
equip each environment with a performance function that is hidden from the
agent. This allows us to categorize AI safety problems into robustness and
specification problems, depending on whether the performance function
corresponds to the observed reward function. We evaluate A2C and Rainbow, two
recent deep reinforcement learning agents, on our environments and show that
they are not able to solve them satisfactorily.
DepthSynth: Real-Time Realistic Synthetic Data Generation from CAD Models for 2.5D Recognition
Authors: Benjamin Planche, Ziyan Wu, Kai Ma, Shanhui Sun, Stefan Kluckner, Terrence Chen, Andreas Hutter, Sergey Zakharov, Harald Kosch, Jan Ernst
Date: 2017-02-27 00:00:00
Tasks: Synthetic Data Generation

Recent progress in computer vision has been dominated by deep neural networks
trained over large amounts of labeled data. Collecting such datasets is however
a tedious, often impossible task; hence a surge in approaches relying solely on
synthetic data for their training. For depth images however, discrepancies with
real scans still noticeably affect the end performance. We thus propose an
end-to-end framework which simulates the whole mechanism of these devices,
generating realistic depth data from 3D models by comprehensively modeling
vital factors e.g. sensor noise, material reflectance, surface geometry. Not
only does our solution cover a wider range of sensors and achieve more
realistic results than previous methods, assessed through extended evaluation,
but we go further by measuring the impact on the training of neural networks
for various recognition tasks; demonstrating how our pipeline seamlessly
integrates such architectures and consistently enhances their performance.
When are epsilon-nets small?
Authors: Andrey Kupavskii, Nikita Zhivotovskiy
Date: 2017-11-28 00:00:00

In many interesting situations the size of epsilon-nets depends only on $\epsilon$ together with different complexity measures. The aim of this paper is to give a systematic treatment of such complexity measures arising in Discrete and Computational Geometry and Statistical Learning, and to bridge the gap between the results appearing in these two fields. As a byproduct, we obtain several new upper bounds on the sizes of epsilon-nets that generalize/improve the best known general guarantees. In particular, our results work with regimes when small epsilon-nets of size $o(\frac{1}{\epsilon})$ exist, which are not usually covered by standard upper bounds. Inspired by results in Statistical Learning we also give a short proof of the Haussler's upper bound on packing numbers.